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Foreword 


Permutations have a remarkably rich combinatorial structure. Part of the rea- 
son for this is that a permutation of a finite set can be represented in many 
equivalent ways, including as a word (sequence), a function, a collection of dis- 
joint cycles, a matrix, etc. Each of these representations suggests a host of nat- 
ural invariants (or “statistics” ), operations, transformations, structures, etc., 
that can be applied to or placed on permutations. The fundamental statis- 
tics, operations, and structures on permutations include descent set (with 
numerous specializations), excedance set, cycle type, records, subsequences, 
composition (product), partial orders, simplicial complexes, probability dis- 
tributions, etc. How is the newcomer to this subject able to make sense of 
and sort out these bewildering possibilities? Until now it was necessary to 
consult a myriad of sources, from textbooks to journal articles, in order to 
grasp the whole picture. Now, however, Miklds Béna has provided us with a 
comprehensive, engaging, and eminently readable introduction to all aspects 
of the combinatorics of permutations. The chapter on pattern avoidance is 
especially timely and gives the first systematic treatment of this fascinating 
and active area of research. 

This book can be utilized at a variety of levels, from random samplings of 
the treasures therein to a comprehensive attempt to master all the material 
and solve all the exercises. In whatever direction the reader’s tastes lead, a 
thorough enjoyment and appreciation of a beautiful area of combinatorics is 
certain to ensue. 


Richard Stanley 


Cambridge, Massachusetts 
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Preface to the First Edition 


A few years ago, I was given the opportunity to teach a graduate combinatorics 
class on a special topic of my choice. I wanted the class to focus on the 
combinatorics of permutations. However, I instantly realized that while there 
were several excellent books that discussed some aspects of the subject, there 
was no single book that would have contained all, or even most, areas that I 
wanted to cover. Many areas were not covered in any book, which was easy to 
understand as the subject is developing at a breathtaking pace, producing new 
results faster than textbooks are published. Classic results, while certainly 
explained in various textbooks of very high quality, seemed to be scattered 
in numerous sources. This was again no surprise; indeed, permutations are 
omnipresent in modern combinatorics, and there are quite a few ways to look 
at them. We can consider permutations as linear orders; we can consider 
them as elements of the symmetric group; we can model them by matrices; 
or by graphs. We can enumerate them according to countless interesting 
statistics; we can decompose them in many ways, and we can bijectively 
associate them to other structures. One common feature of these activities is 
that they all involve factual knowledge, new ideas, and serious fun. Another 
common feature is that they all evolve around permutations, and quite often, 
the remote-looking areas are connected by surprising results. Briefly, they do 
belong to one book, and I am very glad that now you are reading such a book. 


2K OK 


As I have mentioned, there are several excellent books that discuss various 
aspects of permutations. Therefore, in this book, I cover these aspects less 
deeply than the areas that have previously not been contained in any book. 
Chapter 1 is about descents and runs of permutations. While Eulerian num- 
bers have been given plenty of attention during the last 200 years, most of the 
research was devoted to analytic concepts. Nothing shows this better than 
the fact that I was unable to find published proofs of two fundamental results 
of the area using purely combinatorial methods. Therefore, in this chapter, I 
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Xiv Preface to the First Edition 


concentrated on purely combinatorial tools dealing with these issues. By and 
large, the same is true for Chapter 2, whose subject is inversions in permuta- 
tions, and in permutations of multisets. Chapter 3 is devoted to permutations 
as products of cycles, which is probably the most-studied of all areas covered 
in this book. Therefore, while there were many classic results we had to 
include there for the sake of completeness, nevertheless we still managed to 
squeeze in less well-known topics, such as applications of Darroch’s theorem, 
or transpositions and trees. 

The area of pattern avoidance is a young one, and has not been given sig- 
nificant space in textbooks before. Therefore, we devoted two full chapters 
to it. Chapter 4 walks the reader through the quest for the solution of the 
Stanley-Wilf conjecture, ending with the recent spectacular proof of Marcus 
and Tardos for this 23-year-old problem. Chapter 5 discusses aspects of pat- 
tern avoidance other than upper bounds or exact formulae. Chapter 6 looks 
at random permutations and Standard Young Tableaux, starting with two 
classic and difficult proofs of Greene, Nijenhaus, and Wilf. Standard tech- 
niques for handling permutation statistics are presented. A relatively new 
concept, that of min-wise independent families of permutations, is discussed 
in the Exercises. Chapter 7, Algebraic Combinatorics of Permutations, is the 
one in which we had to be very selective. Each of the three sections of that 
chapter covers an area that is sufficiently rich to be the subject of an entire 
book. Our goal with that chapter is simply to raise interest in these topics 
and prepare the reader for the more detailed literature that is available in 
those areas. Chapter 8 is about combinatorial sorting algorithms, many of 
which are quite recent. This is the first time many of these algorithms (or at 
least, most aspects of them) are discussed in a textbook, so we treated them 
in depth. 

Besides the Exercises, each chapter ends with a selection of Problems Plus. 
These are typically more difficult than the exercises, and they are meant to 
raise interest in some questions for further research, and to serve as reference 
material of what is known. Some of the Problems Plus are not classified 
as such because of their level of difficulty, but because they are less tightly 
connected to the topic at hand. A solution manual for the even-numbered 
Exercises is available for instructors teaching a class using this book, and it 
can be obtained from the publisher. 


Preface to the Third Edition 


It has been nine years since the second edition of Combinatorics of Permu- 
tations was published. Many areas of the subject went through significant 
progress during those years. 

The youngest area, permutation patterns, that is the content of our Chapters 
4 and 5, was a major contributor to that progress. Several new methods were 
discovered to prove upper bounds on the size of some permutation classes, 
most of all the extremely challenging class of 1324-avoiding permutations. 
We know now that most principal permutation classes have a nonrational 
generating function, but a conjecture on which patterns of a given size are the 
easiest to avoid proved to be false. Records have fallen in just about every 
version of the superpattern problem. 

The related area of stack sorting also has much stronger, and more numer- 
ous, results than it had nine years ago. This also means that some of the 
long-standing conjectures of the field are very likely to be false, though we do 
not always know which ones. Chapter 8 describes some of this progress, and 
the new open problems that it brought. 

The enumeration of vertices of a given rank in many varieties of rooted 
trees has been the subject of vigorous research in the last decade. As many 
of these tree varieties correspond to some kind of permutations, these results 
will appear throughout the book. Section 6.3 is entirely devoted to this topic. 

As the Combinatorics of Permutations keeps expanding, books about the 
subject must become more and more selective. While it is impossible to cover 
all, or even, most, of the important results in the field, we tried to discuss at 
least those that we find particularly rewarding, and that we want the readers 
to enjoy as well. 

Gainesville, FL 
February 2022 
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No Way around It. Introduction. 


This book is devoted to the study of permutations. While the overwhelming 
majority of readers already know what they are, we are going to define them 
for the sake of completeness. Note that this is by no means the only definition 
possible. 


DEFINITION 0.1 A linear ordering of the elements of the set [n] = 
{1,2,3,--- ,n} is called a permutation, or, if we want to stress the fact that 
it consists of n entries, an n-permutation. 


In other words, a permutation lists all elements of [n] so that each element 
is listed exactly once. 


Example 0.2 
If n = 3, then the n-permutations are 123, 132, 218, 231, 312, 321. [] 


There is nothing magic about the set [n]; other sets having n elements 
would be just as good for our purposes, but working with [n] will simplify 
our discussion. In Chapter 2, we will extend the definition of permutations 
to multisets, and in Chapter 3, we will consider permutations from a different 
perspective. The set of all n-permutations will be denoted by S,, and the 
reason for that will become clear in Chapter 3. 

For now, we will denote an n-permutation by p = p)p2--- pn, with p; being 
the ith entry in the linear order given by p. 

The following simple statement is probably the best-known fact about per- 
mutations. 


PROPOSITION 0.3 


The number of n-permutations is nl. 


PROOF ~~ When building an n-permutation p = p,p2--- pn, we can choose 
n entries to play the role of p;, then n — 1 entries for the role of pz, and so 
on. 


We promise the rest of the book will be less straightforward. 
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In One Line and Close. Permutations as 
Linear Orders. 


1.1 Descents 


The “most orderly” of all n-permutations is obviously the increasing permuta- 
tion 123---n. All other permutations have at least some “disorder” in them; 
for instance, it happens that an entry is immediately followed by a smaller 
entry in them. This simple phenomenon is at the center of our attention in 
this Section. 


1.1.1 Definition of Descents 


DEFINITION 1.1 Let p= pip2-:-:pn be a permutation, and leti<n 
be a positive integer. We say that i is a descent of p if p; > pi4i1- Similarly, 
we say that i is an ascent of p if pi < pi41- 


Example 1.2 
Let p = 3412576. Then 2 and 6 are descents of p, while 1, 3, 4, and 5 are 
ascents of p. 


Note that the descents denote the positions within p, and not the entries of 
p. The set of all descents of p is called the descent set of p and is denoted by 
D(p). The cardinality of D(p), that is, the number of descents of p, is denoted 
by d(p), though certain authors prefer des(p). 

This very natural notion of descents raises some obvious questions for the 
enumerative combinatorialist. How many n-permutations are there with a 
given number of descents? How many n-permutations are there with a given 
descent set? If two n-permutations have the same descent set, or the same 
number of descents, what other properties do they share? 

The answers to these questions are not always easy, but are always inter- 
esting. We would like to start with the problem of finding the number of 
permutations with a given descent set S. However, it turns out that it is even 
easier to find the number of permutations whose descent set is contained in 
S, so we start with that result. 
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LEMMA 1.3 
Let S = {81,89,--+ ,8~} C [n—1], where the 81 < sg < +++ < sx, and let a(S) 
be the number of n-permutations whose descent set is contained in S. Then 


the identity 
sas co ce ome aa 
S1 $2 — S1 $3 — 82 nm— Sk 
holds. 


PROOF ~The crucial idea of the proof is the following. We arrange our n 
entries into k+1 segments so that the first 2 segments together have s; entries 
for each 7. Then, within each segment, we put our entries in increasing order. 
Then the only places where the resulting permutation has a chance to have a 
descent is where two segments meet, that is, at 51, s2,--- ,s,%. Therefore, the 
descent set of the resulting permutation is contained in S. 

How many ways are there to arrange our entries in these segments? The 
first segment has to have length s;, and therefore the entries that go there 
can be chosen in ea ways. The second segment has to be of length s2 — s1, 
and has to be disjoint from the first one. Therefore, its entries can be chosen 
in aed) ways. In general, the ith segment must have length s; — s;_1 if 
i < k +1, and its entries have to be chosen from the remaining n — s;_1 


entries, in (aoe) ways. There is only one choice for the last segment as all 


remaining n — s, entries have to go there. This completes the proof. | 


Now we are in a position to state and prove the formula for the number of 
n-permutations with a given descent set. 


THEOREM 1.4 
Let S C [n—1]. Then the number of n-permutations with descent set S' is 


(8) = 93 (-1)!®-Ta(z). (1.1) 


TCS 


PROOF This is a direct conclusion of the Principle of Inclusion and 
Exclusion. (See any textbook on introductory combinatorics, such as [74], for 
this principle.) Note that permutations with a given h-element descent set 
H C S are counted ap, = ran Oy lace = (1+(—1))!5—#! times on the 
right-hand side of (1.1). The value of a), is 0 except when |S — H| = 0, that 
is, when S = H. So the right-hand side counts precisely the permutations 
with descent set S. 
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1.1.2 Eulerian Numbers 


Let A(n,k) be the number of n-permutations with k — 1 descents. You may 
be wondering what the reason for this shift in the parameter k is. If p has 
k —1 descents, then p is the union of k increasing subsequences of consecutive 
entries. These are called the ascending runs of p. (Some authors call them just 
“runs,” others call something else “runs.” This is why we add the adjective 
“ascending” to avoid confusion.) Also note that in some papers, A(n,k) is 
used to denote the number of permutations with k descents. 


Example 1.5 
The three ascending runs of p = 2415367 are 24, 15, and 367. 0 


Example 1.6 

There are four permutations of length three with one descent, namely 132, 213, 
231, and 312. Therefore, A(3,2) = 4. Similarly, A(3,3) = 1 corresponding to 
the permutation 321, and A(3,1) = 1, corresponding to the permutation 123. 


O 


Thus the permutations with k ascending runs are the same as permutations 
with k — 1 descents, providing one answer for the notation A(n,k). We note 
that some authors use the notation (7) for A(n, k). 

The numbers A(n,k) are called the Eulerian numbers, and have several 
beautiful properties. Several authors provided extensive reviews of this field, 
including Leonard Carlitz [110], Dominique Foata and Marcel-Paul 
Schiitzenberger [172], Donald E. Knuth [230], and Charalambos Charalam- 
bides [109]. The most recent, and most comprehensive, coverage of the sub- 
ject is the book [261] by T. Kyle Petersen. In our treatment of the Eulerian 
numbers, we will make an effort to be as combinatorial as possible, and avoid 
the analytic methods that probably represent a majority of the available lit- 
erature. We start by proving a simple recurrence relation. 


THEOREM 1.7 
For all positive integers k and n satisfying k <n, the identity 
A(n,k+1)=(k+1)A(n—-1,k +1) + (n-—k)A(n -1,k) 
holds. 
PROOF There are two ways we can get an n-permutation p with k 
descents from an (n — 1)-permutation p’ by inserting the entry n into p’. 


Either p’ has k descents, and the insertion of n does not form a new descent, 
or p’ has k — 1 descents, and the insertion of n does form a new descent. 
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In the first case, we have to put the entry n at the end of p’, or we have 
to insert n between two entries that form one of the k descents of p’. This 
means we have k + 1 choices for the position of n. As we have A(n—1,k+1) 
choices for p’, the first term of the -hand side is explained. 

In the second case, we have to put the entry n at the front of p’, or we have 
to insert n between two entries that form one of the (n—2)—(k-1) =n—k-1 
ascents of p’. This means that we have n — k choices for the position of n. As 
we have A(n — 1,k) choices for p’, the second part of the right-hand side is 


explained, and the theorem is proved. | 


We note that A(n,k + 1) = A(n,n —k); in other words, the Eulerian 
numbers are symmetric. Indeed, if p = pypo---pyn has k descents, then its 
reverse p” = PnPn—1°+: pi has n — k — 1 descents. 

The following theorem shows some additional significance of the Eulerian 
numbers. In fact, the Eulerian numbers are sometimes defined using this 
relation. 


THEOREM 1.8 
Set A(0,0) =1, and A(n,0) =0 forn > 0. Then for all nonnegative integers 
n, and for all complex numbers z, the equality 


= 3 A(n, k) (; 7 s s (1.2) 


k=1 


holds. 


Example 1.9 
Let n = 3. Then we have A(3,1) = 1, A(83,2) = 4, and A(3,3) = 1, enumer- 
ating the sets of permutations {123}, {132, 213, 231,312}, and {321}. And 


indeed, 
3 e+2 z+l z 
— 4 . 
es) 3) +G) 


PROOF (of Theorem 1.8) Let us assume first that z is a positive integer. 
Then the left-hand side counts the n-element sequences in which each digit 
comes from the set [z]. We will show that the right-hand side counts these 
same sequences. Let a = a1d2---Gp be such a sequence. Rearrange entries 
of a into a nondecreasing order a’ = aj, < ai, <--- < a;,, with the extra 
condition that identical digits appear in a’ in the increasing order of their 
indices. Then i = tji2---tn = p(a) is an n-permutation that is uniquely 
determined by a. Note that 7; tells from which position of a the first entry of 
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a’ comes, 72 tells from which position of a the second entry of a’ comes, and 
so on. 

For instance, if a = 311243, then the rearranged sequence is a’ = 112334, 
leading to the permutation 7 = 234165. 

If we can show that each permutation i having k — 1 descents is obtained 
from exactly ce) sequences a in this way, then we will have proved the 
theorem. 

The crucial observation is that if a;, = a;,,, in a’, then i; < ij41 in i. 
Taking contrapositives, if 7 is a descent in 7 = 7122--+tn, then Qi; < Qijgr- 
This means that the sequence a’ has to be strictly increasing whenever j is a 
descent of 7172---+%,. The reader should verify that in our running example, 7 
has descents at 3 and 5, and indeed, a’ is strictly increasing in those positions 
(but not only there). 

How many sequences a can lead to the permutation 7 = 234165? It follows 
from the above argument that in sequences with that property, we must have 


1 < a2 < a3 < a4 <a, < a6 <5 <@, 


as strict inequality is required in the third and fifth positions. The above 
chain of inequalities is obviously equivalent to 


1l<ag<ag+1<a4+2<a,4+2<ag+3<a5+3< 24+3, 


and, therefore, the number of such sequences is clearly 


z+3 
(s") 

So this is the number of sequences a for which a’ = 234165. Generalizing this 
argument for any n and for permutations 7 with k—1 descents, we get that each 
n-permutation with k — 1 descents will be obtained from (Cae ey) = 
(Ae) sequences. 

If z is not a positive integer, note that the two sides of the equation to be 
proved can both be viewed as polynomials in the variable z. As they agree for 
infinitely many values (the positive integers), they must be identical. 


Exercise 7 gives a more mechanical proof that simply uses Theorem 1.7. 


COROLLARY 1.10 
For all positive integers n, the identity 


z= Ag n(77874) 


k=1 


holds. 
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PROOF ~ Replace z by —z in the result of Theorem 1.8, to get 


2-1)" = dAtn k) & a 7 ‘), 


Now note that 
(*tP as z+n—k)(-z+n—k-1)---(-z+1-k) 


: =can(7Fh~), 


Comparing these two identities yields the desired result. | 


n! 


The obvious question that probably crossed the mind of the reader by now 
is whether there exists an explicit formula for the numbers A(n,k). The 
answer to that question is in the affirmative, though the formula contains a 
summation sign. This formula is more difficult to prove than the previous 
formulae in this section. 


THEOREM 1.11 
For all nonnegative integers n and k satisfying k <n, the identity 


k 
Atak) = o-1y'(" Fe 9" (1.3) 


z a 
1=0 


holds. 


While this theorem is a classic (it is more than 100 years old), in 2003 we 
could not find an immaculately direct proof for it in the literature. Proofs 
we did find used generating functions, or manipulations of double sums of 
binomial coefficients, or inversion formulae to obtain (1.3). Therefore, we 
solicited simple, direct proofs at the problem session of the 15th Formal Power 
Series and Algebraic Combinatorics conference, which took place in Vadstena, 
Sweden. The proof we present here was contributed by Richard Stanley. A 
similar proof was proposed by Hugh Thomas. 


PROOF (of Theorem 1.11) Let & > 0, and let us write down k — 1 bars 
with & compartments in between. (For k = 0, the statement of the theorem 
is obvious.) Place each element of [n] in a compartment. There are k” ways 
to do this, the term in the above sum indexed by i = 0. Arrange the numbers 
in each compartment in increasing order. For example, if k = 4 and n = 9, 
then one arrangement is 


237||19|4568. (1.4) 
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Ignoring the bars, we get a permutation (in the above example, it is 237194568) 
with at most k — 1 descents. 

There are several issues to take care of. There could be empty compart- 
ments, or there could be neighboring compartments with no descents in be- 
tween. We will show how to sieve out permutations having either of these 
problems, and therefore, less than k — 1 descents, at the same time. 

Let us say that a bar is a wall if it is not immediately followed by another 
bar. Let us say that a wall is extraneous if by removing it we still get a legal 
arrangement, that is, an arrangement in which each compartment consist of 
integers in increasing order. 

For instance, in (1.4), the second bar is an extraneous wall. Our goal is to 
enumerate the arrangements with no extraneous walls, as these are clearly in 
bijection with permutations with k — 1 descents. 

In order to do this, we will apply the Principle of Inclusion and Exclusion. 
Let us call the spaces between consecutive entries of a permutation, as well 
as the space preceding the first entry and the space following the last entry a 
position. So we associate n+ 1 positions to an n-permutation. Let S C [n+1], 
and let Ags be the set of arrangements in which there is an extraneous wall in 
each position belonging to S. 

Let i < k —1 be the size of S. Then we claim that 


|As| = (k- 8)". 


In order to see this, first take any legal arrangement that contains k—7—1 bars. 
There are (k — 7)” such arrangements as we can proceed from the entry 1 to 
the entry n in an increasing order, and choose a compartment for each entry. 
Now insert i extra bars by inserting one to each position that belongs to S. (If 
there is already a bar in such a position, then put the new bar immediately 
on the right of that bar.) This results in an arrangement that belongs to 
Ag. Conversely, each arrangement belonging to Ag will be obtained exactly 
once in this way. Indeed, if a € Ag, then removing one bar from each of 
the 7 positions that belong to S', we get the unique original arrangement with 
k —i—1 bars that leads to a. 

As there are (ee) choices for the set S, the proof of our theorem is now 


immediate by the Principle of Inclusion and Exclusion. | 


For the sake of completeness, we include a more computational proof that 
does not need a clever idea as the previous one did. 
First, we recall a lemma from the theory of binomial coefficients. 


LEMMA 1.12 
[Chu - Vandermonde Convolution Formula] Let x and y be real numbers, and 
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let z be a positive integer. Then the identity 


ee -> (i) i 


holds. 


PROOF _ Let us assume first that x and y are positive integers. Then the 
left-hand side enumerates the z-element subsets of the set [x + y], while the 
right-hand side enumerates these same objects, according to the size of their 
intersection with the set [2]. 

For general 2 and y, note that both sides can be viewed as polynomials 
in x and y, and they agree for infinitely many values (the positive integers). 

Therefore, they have to be identical. 


PROOF (of Theorem 1.11) As a first step, consider formula (1.2) with 
x = 1, then with x = 2, and then for x = 7 fori < k. We get 


and so on, the Ath equation being 


h-1 . 
vf(n+gj-1 
Ar = - : 
>  A(n, k ( 7 : (1.5) 
j=0 
and the last equation being 
k-1 . 
fe ~fn+tg—-—1 
ke = ) tn. i)( if ) (1.6) 
j=0 


We will now add certain multiples of our equations to the last one, so that 
the left-hand side becomes the right-hand side of formula (1.3) that we are 
trying to prove. 

To start, let us add (—1)("f") times the (k — 1)st equation to the last one. 
Then add ("5") times the (k — 2)nd equation to the last one. Continue this 
way, that is, in step 7, add (—1)‘("1") times the (k —1)th equation to the last 
one. This gives us 


d-n(" P)e-ar= 40.03 Gee Gale. 


(1.7) 
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The left-hand side of (1.7) agrees with the right-hand side of (1.3). There- 
fore, (1.3) will be proved if we can show that the coefficient a(n, 7) of A(n, J) 
on the right-hand side above is 0 for 7 < k. It is obvious that a(n, k) = 1 as 
A(n,k) occurs in the last equation only. 

Set b= k—j. Then a(n,k) can be transformed as follows. 


a (n n—-i 
a(nk) = Dei ae a. 


Recalling that for a positive real number x, we have (~,”) = (**¢~")(-1)*, 
and noting that (—1)’ = (—1)°-?", this yields 


conan TNL 


i=0 


EMC C)-ECICI-0-4 


where the last step holds as b = k—Jj > 0, and the next-to-last step is a direct 
application of Lemma 1.12. 
This shows that the right-hand side of (1.7) simplifies to A(n, &), and proves 


our theorem. | 
1.1.3 Stirling Numbers and Eulerian Numbers 


DEFINITION 1.13 _ A partition of the set [n] into r blocks is a distri- 
bution of the elements of [n] into r disjoint nonempty sets, called blocks, so 
that each element is placed into exactly one block. 


In Section 2.1, we will define the different concept of partitions of an integer. 
If there is a danger of confusion, then partitions of the set [n] will be called 
set partitions, to distinguish them from partitions of the integer n. 


Example 1.14 
Let n = 7 and r =4. Then {1, 2,4}, {3,6}, {5}, {7} is a partition of [7] into 
four blocks. 


Note that neither the order of blocks nor the order of elements within each 
block matters. That is, {4,1,2}, {6,3}, {5}, {7} and {4, 1,2}, {6,3}, {7}, {5} 
are considered the same partition as the one in Example 1.14. 


DEFINITION 1.15 The number of partitions of [n| into k blocks is 
denoted by S(n,k) and is called a Stirling number of the second kind. 
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n=0 il 

n=1 0 1 

n=2 0 1 1 

n=3 0 1 3 1 

n=4 0) 1 7 6 1 

n=5 0 1 15 25 10 1 


FIGURE 1.1 
The values of S(n, k) for n < 5. Note that the Northeast-Southwest diagonals 
contain values of S(n,k) for fixed k. Row n starts with S(n,0). 


By convention, we set S(n,0) = 0 if n > 0, and S(0,0) = 1. The next 
chapter will explain what the Stirling numbers of the first kind are. 


Example 1.16 
The set [4] has six partitions into three blocks, each consisting of one doubleton 
and two singletons. Therefore, S$(4,3) = 6. 


Whereas Stirling numbers of the second kind do not directly count permu- 
tations, they are inherently related to two different sets of numbers that do. 
One of them is the set of Eulerian numbers, and the other one is the afore- 
mentioned set of Stirling numbers of the first kind. Therefore, exploring some 
properties of the numbers S(n,k) in this book is well-motivated. See Figure 
1.1 for the values of S(n,k) for n < 5. 

See Exercises 8 and 14 for two simple recurrence relations satisfied by the 
numbers S(n,k). It turns out that an explicit formula for these numbers can 
be proved without using any recurrence relations. 


LEMMA 1.17 
For all positive integers n and r, the identity 


holds. 


PROOF An ordered partition of [n] into r blocks is a partition of [n] into 
r blocks in which the set of blocks is totally ordered. So {1,3}, {2,4} and 
{2,4}, {1,3} are different ordered partitions of [4] into two blocks. Note that 
an ordered partition of [n] into r blocks is just the same as a surjection from [n] 
to [r]. In order to enumerate all such surjections, let A; be the set of functions 
from [n] into [r] whose image does not contain 7. The function f : [n] > [r] is 
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a surjection if and only if it is not contained in Ay U Ap U---UA,, and our 
claim can be proved by a standard application of the Principle of Inclusion 
and Exclusion. 


Stirling numbers of the second kind and Eulerian numbers are closely re- 
lated, as shown by the following theorem. 


THEOREM 1.18 
For all positive integers n and r, the identity 


S(n,r) = 5 AGA) te (1.8) 
holds. 


PROOF = Multiplying both sides by r! we get 


rlS(n,r) = s A(n, k) ; - ‘) ; 


Here the left-hand side is obviously the number of ordered partitions of [n] 
into r blocks. We will now show that the right-hand side counts the same 
objects. Take a permutation p counted by A(n,k). The & ascending runs of 
p then naturally define an ordered partition of [n] into & parts. If k = r, then 
there is nothing left to do. If k < r, then we will split up some of the ascending 
runs into several blocks of consecutive elements, in order to get an ordered 
partition of r blocks. As we currently have k blocks, we have to increase the 
number of blocks by r—k. This can be achieved by choosing r—k of the n—k 
“gap positions” (gaps between two consecutive entries within the same block). 

This shows that we can generate >)» A(n, nC.) ordered partitions of 
[n] that consist of r blocks each by the above procedure. It is straightforward 
to show that each such partition will be obtained exactly once. Indeed, if 
we write the elements within each block of the partition in increasing order, 
we can just read the entries of the ordered partition left to right and get the 
unique permutation having at most r ascending runs that led to it. We can 
then recover the gap positions used. This completes the proof. 


Inverting this result leads to a formula expressing the Eulerian numbers by 
the Stirling numbers of the second kind. 


COROLLARY 1.19 
For all positive integers n and k, the identity 


: n—-?T 
A(n,k) = S> S(n,r)r! 6 7 ") (1 (1.9) 
holds. = 
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PROOF _ Let us consider formula (1.8) for each r < k, and multiply each 
equality by r!. We get the equations 


LU. $(n,1) = A(n, n("5 2 


2!. S(n,2) = A(n,1) (" ; ‘) + A(n,2) G - *) ' 


the equation for general r being 


rl S(n,r) => A(n,i) (a (1.10) 
i=1 
and the last equation being 
s n—-4t 
k!- S(n,k) = aeno("7"), (1.11) 
i=1 


Our goal is to eliminate each term from the right-hand side of (1.11), except 
for the term A(n, Bees) = A(n,k). We claim that this can be achieved by 
multiplying (1.10) by (—1)*-"("—"), doing this for all r € [k — 1], then adding 
these equations to (1.11). 

To verify our claim, look at the obtained equation 


Sseunrt-yr (227) = Seve") Sawa"), 


rT 


(1.12) 
or, after changing the order of summation, 
E n-?Tr " n-i\< n-r 
W—-1)F-"( J = SY A(n,i)[ Sr 
DS(on Gs) » co) anna D3 i) 
(1.13) 


whose left-hand side is identical to the right-hand side of (1.9). 

It is obvious that the coefficient of A(n, k) on the right-hand side is (2) = 
1. Therefore, our statement will be proved if we can show that the coefficient 
t(n,2) of A(n,7) in the last expression is equal to zero if i < k. 


Note that ("7") =0 ifr <i. Therefore, for any fixed i < k, we have 


9-E CDG Dor - EY) 


r=2 y sheen 
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We used Cauchy’s convolution formula (Lemma 1.12) in the last step. This 
proves that if i < k, then A(n,7) vanishes on the right-hand side of (1.13). We 
have discussed that A(n,k) will have coefficient 1 there. (This can be seen 
again by setting & = 7 in the last expression, leading to t(n,7) = tee) = 1.) 
So (1.13) implies the claim of this corollary. | 


1.1.4 Generating Functions and Eulerian Numbers 


There are several ways one can define a generating function whose coefficients 
are certain Eulerian numbers. Let us start with a “horizontal” version. 


DEFINITION 1.20 For all nonnegative integers n, the polynomial 
An(z) = 5° A(n, k)z* 
k=1 


is called the nth Eulerian polynomial. 


The Eulerian polynomials have several interesting properties that can be 
proved by purely combinatorial means. We postpone the study of those prop- 
erties until the next subsection. For now, we will explore the connection 
between these polynomials and some infinite generating functions. 


THEOREM 1.21 
For all positive integers n, the nth Eulerian polynomial has the alternative 
description 

Aa" yore. 


i>0 


Note that Euler first defined the polynomials A,,(z) in the above form. 


Example 1.22 
For n = 1, we have 


and for n = 2, we have 


Ao(z) = (1-2)? $02! = (1-2). (Sta) =z+2, 


i>0 
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PROOF (of Theorem 1.21) Let us use (1.3) to write the Eulerian poly- 


nomials as 
S*A(n, k)2* => SO (1 & Jt ~iyr2k 
k=1 k=10<i<k e 
” _,(n+l)\. k 
= S- yy Git i ( ‘en 
k=1 \0<i<k k-4 


Changing the order of summation, and noting that the sum in parentheses, 
being equal to A(n,k), vanishes for k > n, we get 


S- jez! : S- 4 =) ae = (1 - zt S- iret. 
i>0 k>i i>0 


It is often useful to collect all Eulerian numbers A(n,k) for all n and all k 
in a master generating function. This function turns out to have the following 
simple form. 


THEOREM 1.23 


Let 64 
r(,z)= >. An, eS 
n>0k>0 
Then the equality 
isis 1-t 
r( 52) a 1 _ tez(i-t) 


holds. 


PROOF _ Using the result of Theorem 1.21, we see that 


n 


r(t,z) = > (—#"*? x int? — = (1-2) Sot S ve ty" 


n>0 i>0 i>0  n>0 


a 1-t 
ox i iz(1-t) _ 
(1 -#) ) tre = een 


i>0 
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n=! 1 

n=2 1 1 

n=3 1 4 1 

n=4 1 11 11 1 
a 1 26 66 26 1 


n=6 1 all 302. 302057 1 


FIGURE 1.2 


Eulerian numbers for n < 6. Again, the NE-SW diagonals contain the values 
of A(n,k) for fixed k. Row n starts with A(n, 1). 


1.1.5 Sequences of Eulerian Numbers 


Let us take a look at the numerical values of the Eulerian numbers for small 
n, and k = 0,1,:--,n—1. The nth row of Figure 1.2 contains the values of 
A(n,k), for l<k<n,up ton=6. 

We notice several interesting properties. As we pointed out before, the 
sequence A(n,k) is symmetric for any fixed n. Moreover, it seems that these 
sequences first increase steadily, then decrease steadily. This property is so 
important in combinatorics that it has its own name. 


DEFINITION 1.24 We say that the sequence of positive real numbers 
Q1,42,°°* , Gy is unimodal if there exists an index k such thatl1<k <n, and 
QS ag°+* Sap 2 Aky1 2+ 2 An. 


The sequences A(n, k) 41<p<n} seem to be unimodal for any fixed n. In fact, 
they seem to have a stronger property. 


DEFINITION 1.25 We say that the sequence of positive real numbers 
@1,02,°** , Gn ts log-concave if ap—1dk41 < az holds for all indices k. 


PROPOSITION 1.26 
If the sequence a1, 42,°-* ,Qn of positive real numbers is log-concave, then it 
is also unimodal. 


PROOF ~ The reader should find the proof first, then check the proof that 
we provide as a solution for Exercise 5. 


The conjecture suggested by our observations is in fact correct. This is the 
content of the following theorem. 
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THEOREM 1.27 
For any positive integer n, the sequence A(n,k)t1<r<n} of Eulerian numbers 
is log-concave. 


While this result has been known for a long time, it was usually shown 
as a corollary to a stronger, analytical result that we will discuss shortly, in 
Theorem 1.34. Direct combinatorial proofs of this fact are more recent. The 
proof we present here was given by Bona and Ehrenborg [51] who built on an 
idea of Vesselin Gasharov [189]. 

If a path on a square grid uses steps (1,0) and (0,1) only, we will call it a 
northeastern lattice path. 

Before proving the theorem, we need to set up some tools, which will be 
useful in the next section as well. We will construct a bijection from the set 
A(n,k) of n-permutations with k descents onto that of labeled northeastern 
lattice paths with n edges, exactly k of which are vertical. (Note the shift in 
parameters: |A(n,k)| = A(n,k&+ 1), but this will not cause any confusion.) 

If a path on a square grid uses steps (1,0) and (0,1) only, we will call it a 
northeastern lattice path. 

Let P(n) be the set of labeled northeastern lattice paths that have edges 
@1,Q2,...,@,, and that corresponding positive integers e€1, €2,...,€n as labels, 
so that the following hold: 


(i) the edge a, is horizontal and e; = 1, 


(ii) if the edges a; and aj, are both vertical, or both horizontal, then 
e; 2 €i41, 


(iii) if a; and aj41 are perpendicular to each other, then e; + e441 <i4+1. 


The starting point of a path in P(n) has no additional significance. Let 
P(n, k) be the set of all lattice paths in P(n) which have & vertical edges, and 
let P(n, k) = |P(n,k)]. 


PROPOSITION 1.28 
The following two properties of paths in P(n) are immediate from the defini- 
tions. 


e For alli > 2, the inequality e; < i—1 holds. 


e Fix the label e;. If e;4, can take value v, then it can take all positive 
integer values w < v. 


Also note that all restrictions on e;;; are given by e;, independently of 
preceding e;, 7 < i. Now we are going to explain how we will encode our 
permutations by these labeled lattice paths. 
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LEMMA 1.29 

The following description defines a bijection from S(n) onto P(n), where 
S(n) is the set of all n-permutations. Let p € S(n). To obtain the edge a; and 
the label e; for2<i<n, restrict the permutation p to the i first entries and 
relabel the entries to obtain a permutation q = q1--- qi of [i]. Then proceed as 
follows. 


1. If the position i —1 is a descent of the permutation p (equivalently, of 
the permutation q), let the edge a; be vertical and the label e; be equal 
to qi- 


2. If the position i—1 is an ascent of the permutation p, let the edge a; be 
horizontal and the label e; bei +1— qG. 


Moreover, this bijection restricts naturally to a bijection between A(n, k) 
and P(n,k) forO<k<n-1. 


PROOF 

The described map is clearly injective. Let us assume that 7 — 1 and i are 
both descents of the permutation p. Let q, respectively r, be the permutation 
when restricted to the i, respectively 7+ 1, first elements. Observe that q; is 
either r; or r; — 1. Since r; > ri41 we have q; > ri41 and condition (ii) is 
satisfied in this case. By similar reasoning, the three remaining cases (based 
on i—1 and i being ascents or descents) are shown, hence the map is into the 
set P(n). 

To see that this is a bijection, we show that we can recover the permutation 
p from its image. To that end, it is sufficient to show that we can recover pn, 
and then use induction on n for the rest of p. To recover p, from its image, 
simply recall that p, is equal to the label @ of the last edge if that edge is 
vertical, and to n+ 1-— ¢ if that edge is horizontal. Conditions (ii) and (iii) 
assure that this way we always get a number between 1 and n for pn. 


See Figure 1.3 for an example of this bijection. 
Now we are in position to prove that the Eulerian numbers are log-concave. 


PROOF (of Theorem 1.27). We construct an injection 
®:P(n,k—1) x P(n,k +1) — P(n,k) x P(n,k). 


This injection ® will be defined differently on different parts of the domain. 
Let (P,Q) € P(n,k-—1) x P(n,k +1). Place the initial points of P and 
Q at (0,0) and (1,—1), respectively. Then the endpoints of P and Q are 
(n—k+1,k—1) and (n—k,k), respectively, so while Q starts “below” P, it 
ends “above” P. 
Let X be the first (most southwestern) common point of P and Q. It then 
follows that P arrives at X by an east step, and Q arrives at X by a north 
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FIGURE 1.3 
The image of the permutation 243165. 


step. We will now show how to proceed if neither P nor Q changes directions 
at X, that is, P leaves X by an east step, and Q leaves X by a north step. 
The other cases are very similar and are left to the reader. Essentially, in all 
of the other cases, one of the transformations discussed below will have the 
desired effect when applied appropriately. See Problem Plus 17 for details. 

Decompose P = P; U Pp and Q = Q; U Qa, where P, is a path from (0,0) 
to X, P, is a path from X to (n—k,k), Qi is a path from (1,—1) to X, 
and @2 is a path from X to (n—k+1,k—1). Let a,b,c,d be the labels of 
the four edges adjacent to X as shown in Figure 1.5, the edges AX and XB 
originally belonging to P and the edges CX and XD originally belonging to 
Q. Then by condition (ii) we have a > b and c > d. Let P’ = P; UQ2 and let 
Q’=Q, UP». 


1. If P’ and Q’ are valid paths, that is, if their labeling fulfills conditions 
(i)—(iii), then we set ®(P,Q) = (P’, Q’). See Figure 1.4 for this construc- 
tion. This way we have defined ® for pairs (P,Q) € P(n,k) x P(n,k) 
in which a+d <i and 6+c <i, where i —1 is the sum of the two coor- 
dinates of X. We also point out that we have not changed any labels, 
therefore, in (P’,Q’) we still have a > b and c > d, though that is no 
longer required, as the edges in question are no longer parts of the same 
path. 


It is clear that ®(P,Q) = (P’,Q') € P(n,k) x P(n,k), (in particular, 
(P’,Q’) belongs to the subset of P(n,k) x P(n,k) consisting of inter- 
secting pairs of paths), and that ® is one-to-one. 


2. There are pairs (P,Q) € P(n,k —1) x P(n,k +1) for which ®(P, Q) 


In One Line and Close. Permutations as Linear Orders. 


4 
ee 
ee 
e ee 
2) 2) 
| rer | 
@ 


1 1 
Oe ee 
FIGURE 1.4 
The new pair of paths. 
’® 
: 
A, X! » 3B 
e', @ © 
Q 
di 
c®@ 


FIGURE 1.5 
Labels around the point X. 
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New labels around the point X. 


cannot be defined in the way it was defined in the previous case, that is, 
when either a+d> ior b+c>i holds. The reader is invited to verify 
that such pairs (P, Q) actually exist. One example is when P is the path 
belonging to the permutation 1237654, and Q is the path belonging to 
the permutation 4567123. 


Change the label of the edge AX to i — c and change the label of the 
edge CX to i—a as seen in Figure 1.6, then proceed as in the previous 
case to get ®(P,Q) = (P’, Q’), where P’ = P, UQ2 and Q’ = Qi U Py. 


We claim that P’ and Q’ are valid paths. Indeed we had at least one of 
a+d>iandb+c>i,so we must havea+c>iasa>bandc>d. 
Therefore, 1 — a < c and 1 —c < a, so we have decreased the values of 
the labels of edges AX and C'X, and that is always possible as shown in 
Proposition 1.28. Moreover, no constraints are violated in P’ and Q’ by 
the edges adjacent to X asi—c+d<iandi—a+b <i. It is also clear 
that ® is one-to-one on this part of the domain, too. Finally, we have 
to show that the image of this part of the domain is disjoint from that 
of the previous part. This is true because in this part of the domain 
we have at least one of a+d>iand 6+ c >i, that is, at least one of 
i—c< bandi—a < d, so in the image, at least one of the pairs of edges 
AX,XB and CX, XD does not have the property that the label of the 
first edge is at least as large as that of the second one. And, as pointed 
out in the previous case, all elements of the image of the previous part 
of the domain do have that property. 


Given ®(P,Q) = (P’, Q’), the vertex X can be uniquely determined as the 


most southwestern point of the intersection of P’ and Q’. 


It then follows that the map ® we created is an injection. This proves the 


inequality 


A(n,k —1)A(n,k +1) < A(n,k)?, 


so our theorem is proved. | 
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There is a property of sequences of positive real numbers that is even 
stronger than log-concavity. 


DEFINITION 1.30 = Let aj,a2,--- ,an be a sequence of positive real 
numbers. We say that this sequence has real roots only or real zeros only if 
the polynomial S>*_, a;z* has real roots only. 


We note that sometimes the sequence can be denoted ag, a1,:-- ,@n, and 
sometimes it is better to look at the polynomial $>;"_, aiz’ (which, of course, 
has real roots if and only if 079 ajz‘t! does). 


Example 1.31 
For all positive integers n, the sequence ag, @1,°-- ,@,, defined by a; = Cy has 
real zeros only. 


SOLUTION We have S77.) giz’ = Oj-g (") 2’ = (1 + 2)", 80 all roots 


4 


of our polynomial are equal to —1. 


Having real zeros is a stronger property than being log-concave, as is shown 
by the following theorem of Newton. 


THEOREM 1.32 


If a sequence of positive real numbers has real roots only, then it is log-concave. 


PROOF Let ao, ai,--- , an be our sequence, and let P(z) = 779 an2”*. 
Then for all roots (z, y) of the polynomial Q(x, y) = Ty anz"y”*, the ratio 
(z/y) must be real. (Otherwise z/y would be a non-real root of P(z)). There- 
fore, by Rolle’s Theorem, this also holds for the partial derivatives 0Q/0z and 
dQ/Ody. Iterating this argument, we see that the polynomial 0°+°Q/dz%dy? 
also has real zeros, if a+b <n-— 1. In particular, this is true in the special 
case when a = 7 —1, and b=n-—j —1, for some fixed 7. This implies that 
the quadratic polynomial R(z,y) = 0"~-2Q/0z)—tdy"-4—* has real roots only, 
and therefore the discriminant of R(z,y) is non-negative. On the other hand, 
we can compute R(z,y) by computing the relevant partial derivatives. Note 
that we only have to look at the values of k ranging from j — 1 to 7+ 1 as all 
other summands of Q(z, y) vanish after derivation. We get 


; . : : ey 
(n—Jj4 1)ly? ta, j!(n—j)!zyt+aj41(n—J I'S 1)!2? 


Nle 


R(z,y) = aj-1(J—-1)! 


As we said, this polynomial has to have a non-negative discriminant, meaning 
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— j+1 p+1 
n—- 
as = ee : Lies ea *Qj-14j41, (1.14) 
J Wes 
which is stronger than our original claim, as > Aj—14;41- | 


The alert reader has probably noticed that by (1.14), a log-concave sequence 
does not necessarily have real zeros only. For instance, the sequence 1, 1, 1 is 
certainly log-concave, but the polynomial P(z) = 1+ z+ z? has two complex 
roots. 

One might ask why we would want to know whether a combinatorially 
defined sequence has real zeros or not. In certain cases, proving the real 
zeros property is the only, or the easiest, way to prove log-concavity and 
unimodality. In some cases, unimodality and log-concavity can be proved by 
other means, but that does not always tell us where the maximum or maxima 
of a given sequence is, or just how many maxima the sequence has. Note that 
a constant sequence is always log-concave, so a log-concave sequence could 
possibly have any number of maxima. The following Proposition shows that 
in a sequence with real zeros only, the situation is much simpler. 


PROPOSITION 1.33 
If the sequence {ax}o<k<n has real zeros only, then it has either one or two 
maximal elements. 


PROOF _ Formula (1.14) shows that in such a sequence, the ratio aj+1/a; 
strictly decreases, so it can be equal to 1 for at most one index 7. | 


Theorem 3.25 will show how to find the maximum (or maxima) of a sequence 
with real zeros. 

The following theorem shows that Eulerian numbers have this last, stronger 
property as well. 


THEOREM 1.34 
For any fixed n, the sequence {A(n,k)}, of Eulerian numbers has real roots 
only. In other words, all roots of the polynomial 


An(z) = 5° A(n, k)z* 
k=1 


are real. 


Recall that the polynomials A,,(z) of Theorem 1.34 are called the Eulerian 
polynomials. This theorem is a classic result, but surprisingly, it is not easy 
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to find a full, self-contained proof for it in the literature. The ideas of the 
proof we present here are due to Herb Wilf and Aaron Robertson. 
PROOF (of Theorem 1.34) Theorem 1.7 implies 

Ag(z) = (2 — 27) Al_1 (2) + nzAn_ilz) (n > 1; Ao(z) = 2). 


Indeed, the coefficient of z* on the left-hand side is A(n,k), while the coeffi- 
cient of z* on the right-hand side is 


kA(n —1,k) — (k—1)A(n—1,k —1) + nA(n—1,k—-1) = 


kA(n—1,k)+(n-—k+1)A(n -—1,k-1) = A(n,k). 


Now note that the right-hand side closely resembles the derivative of a prod- 
uct. This suggests the following rearrangement: 


Apn(z) = 2(1-— zrne {(1— 2)" An-1(z)} (1.15) 


with n > 1 and Ao(z) = z. 

The Eulerian polynomial Aop(z) = z vanishes only at z = 0. Suppose, 
inductively, that A,-1(z) has n — 1 distinct real zeros, one at z = 0, and 
the others negative. From (1.15), or otherwise, A,(z) vanishes at the origin. 
Further, by Rolle’s Theorem, (1.15) shows that A,,(z) has a root between each 
pair of consecutive roots of An_1(z). This accounts for n — 1 of the roots of 
Gn(z). Since we have accounted for all but one root, the remaining last root 
must be real since complex roots of polynomials with real coefficients come in 
conjugate pairs. 


We mention that an elementary survey of unimodal, log-concave, and real- 
roots-only sequences can be found in [75]. The articles [98] and [295] are 
earlier, high-level survey papers. A recent, high-level summary of the subject 
is Chapter 7 of [73], by Petter Brandén. 

Eulerian numbers can count permutations according to properties other 
than descents. Let p = p,p2:::pn be a permutation. We say that 7 is an 
excedance of p if pj > i. (Note that for this definition, it is important to 
require that the entries of p are the elements of [n] and not some other n- 
element set.) 


Example 1.35 
The permutation 24351 has three excedances, 1, 2, and 4. Indeed, pj = 2 > 1, 
po =4>2,andp,=5>4. 


THEOREM 1.36 
The number of n-permutations with k — 1 excedances is A(n,k). 
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We postpone the proof of this theorem until Section 3.3.2, where it will 
become surprisingly easy, due to a different way of looking at permutations. 
However, we mention that if f : S, — N is a function associating natural 
numbers to permutations, then it is often called a permutation statistic. (Re- 
call from the Introduction that S,, denotes the set of all n-permutations.) If 
a permutation statistic f has the same distribution as the statistic “number 
of descents”, that is, if for all k € [n], we have 


|{p © Sn: f(p) =k} | =| {pe Sn: d(p) = k} |, (1.16) 


then we say that f is an Eulerian statistic. So Theorem 1.36 says that “number 
of excedances,” sometimes denoted by exc, is an Eulerian statistic. We will 
see further Eulerian statistics in the Exercises section. 


DT 


1.2 Alternating Runs 


Let us modify the notion of ascending runs that we discussed in the last 
section. Let p = p,p2:-- pn be a permutation. We say that p changes direction 
at position 7 if either pj, < pj > pi4i1, OF Pi-1 > Di < Pi4i1- In other words, 
p changes directions when p; is either a peak or a valley. 


DEFINITION 1.37 We say that p has k alternating runs if there are 
k — 1 indices i so that p changes direction at these positions. 


For example, p = 3561247 has 3 alternating runs as p changes direction 
when i = 3 and when i = 4. A geometric way to represent a permutation 
and its alternating runs by a diagram is shown in Figure 1.7. The alternating 
runs are the line segments (or edges) between two consecutive entries where 
p changes direction. So a permutation has k alternating runs if it can be 
represented by & line segments so that the segments go “up” and “down” 
exactly when the entries of the permutation do. 

The origins of this line of work go back to the nineteenth century. More 
recently, D. E. Knuth [230] has discussed the topic in connection to sorting 
and searching. 

Let G(n, k) denote the number of n-permutations having k alternating runs. 
There are significant similarities between these numbers and the Eulerian 
numbers. For instance, for fixed n, both sequences have real zeros only, and 
both satisfy similar recurrence relations. However, the sequence of the G(n, k) 
is not symmetric. On the other hand, almost half of all roots of the generating 
function Gn(z) = dines, ge) = Ded G(n, k)z* are equal to —1. Here r(p) 
denotes the number of alternating runs of p. 
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FIGURE 1.7 
Permutation 3561247 has three alternating runs. 


First we prove a simple recurrence relation on the numbers G(n,k), which 
was first proved by André in 1883. 


LEMMA 1.38 
For positive integers n and k we have 


G(n, k) = kG(n —1,k) +2G(n —1,k —1) + (n—k)G(n—1,k—2), (1.17) 


where we set G(1,0) =1, and G(1,k) =0 fork > 0. 


PROOF _ Let p be an (n—1)-permutation having k alternating runs, and 
let us try to insert n into p without increasing the number of alternating runs. 
We can achieve that by inserting n at one of k positions. These positions are 
right before the beginning of each descending run, and right after the end of 
each ascending run. This gives us kG(n — 1, k) possibilities. 

Now let g be an (n—1)-permutation having k—1 alternating runs. We want 
to insert n into q so that it increases the number of alternating runs by 1. We 
can achieve this by inserting n into one of two positions. These two positions 
are very close to the beginning and the end of g. Namely, if q starts in an 
ascending run, then insert n to the front of g, and if qg starts in a descending 
run, then insert n right after the first entry of g. Proceed dually at the end 
of the permutation. 


26 Combinatorics of Permutations, Third Edition 


FIGURE 1.8 
The values of G(n,k) for n < 6. The first value of row n is G(n,1). The 
NE-SW diagonals contain the values of G(n,k) for fixed k. 


Finally, let r be an (n — 1)-permutation having k — 2 alternating runs, and 
observe that by inserting n into any of the remaining n— (k —2)-2=n—k 
positions, we increase the number of alternating runs by two. This completes 
the proof. 


The first values of G(n,k) are shown in Figure 1.8 for n < 6. 

Looking at these values of G(n,k), we note they are all even. This is easy 
to explain as p and its reverse always have the same number of alternating 
runs. 

Taking a second look at the polynomials G,(z), we note that G4(z) = 
(z + 1)(102? + 22), and that 


Gs5(z) = 3224 + 5827 + 2827 + 22 = (2 + 1)(322° + 2627 + 22). 


Further analysis shows that Gg(z) and G7(z) are divisible by (z + 1)”, and 
that Gg(z) and Go(z) are divisible by (z + 1)%, and so on. In general, it 
seems that for any positive integer n > 4, the polynomial G,,(z) is divisible 
by (z + 1)\(—?)/2], We will show a recent proof of this fact that comes from 
[82]. 

Let p = pip2-::pn be a permutation. The complement of p is the permuta- 
tion pe =n+1—p,;n+1—p2 ---n+1-—pry. For instance, the complement of 
425613 is 352164. It is clear that p and Dp have the same number of alternating 
runs, since the diagram of p° is just the diagram of p reflected through a hor- 
izontal line. In what follows, we will say flipped instead of reflected through a 
horizontal line. Note that this symmetry implies that all coefficients of G,,(z) 
are even for n > 2. 

Let s be a subsequence of entries in p that are in consecutive positions. Let 
S be set of entries that occur in s, in other words, the underlying set of s. 
Then the complement of s relative to S is the subsequence obtained from s 
so that for each 7, the jth smallest entry of S is replaced by the jth largest 
entry of S. For example, if s=24783, then the complement of s relative to its 
underlying set S' is 84327. 

We will use a similar notion for sets. Let T C U. be finite sets. Let 
T = {ti,to,--- ,t;}, where the ¢; are listed in increasing order. Let us say 
that t; is the a;th smallest element of U. Then the vertical complement of 
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T with respect to U is the set consisting of the a,;th largest elements of U, 
for all i. For instance, if T = {1,4,6}, and U = {1,2,3,4,6,8,9}, then the 
vertical complement of T with respect to U is {3,4,9}. Indeed, T consists of 
the first, fourth, and fifth smallest elements of U, so its vertical complement 
with respect to U consists of the first, fourth, and fifth largest elements of U. 


DEFINITION 1.39 For1l <i<_n, let c be the transformation on 
the set of all permutations p = pip2---DPn that leaves the string p,p2---pi-1 
unchanged, and replaces the string pipi41--*Pn by its complement relative to 
its underlying set. 


Note that ci(p) = p® and c,(p) = p for all p. 


Example 1.40 
Let p = 315462. Then c3(p) = 314526, while cs(p) = 315426. 


PROPOSITION 1.41 
Letn > 4, and let3<i<n-—1. Let p be any permutation of length n. Then 
one of p and c;(p) has exactly one more alternating run than the other. 


PROOF Asc; does not change the number of runs of the string pi po -- + pj—1 
or the number of runs of the string pip;+1---Pn and its image, all changes 
occur within the four-element string p;_-2p;-1pipi+1. There are only 24 pos- 
sibilities for the pattern of these four entries, and it is routine to verify the 
statement for each of the possible 12 pairs. In fact, checking the six pairs in 
which pj_2 < pj_1 is sufficient, for symmetry reasons. 

As it is only the relations among the entries that matter, not the actual 
values of the entries, we can assume that those four entries are 1, 2, 3, and 4. 
Then the six pairs to check are 


e 1234 and 1248, 
e 1324 and 1342, 
e 1423 and 1482, 
e 2134 and 2148, 
e 2314 and 2341, and 
e 3412 and 3421. 


Indeed, each of these pairs consists of two permutations so that one of them 
has one alternating run more than the other. 
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In other words, z™™@) + 2™™”(ci(?)) ig divisible by 1+ z. 
The following lemma is crucial for our purposes. 


LEMMA 1.42 
Let l1<i<g-2< n-—2. Then for all permutations p of length n, the 
identity c;(c;(p)) = cj (ci(p)) holds. 


PROOF Neither c; nor c; acts on any part of the initial segment 
P1p2 ++ pi—1, so that segment, unchanged, will start both ¢;(c;(p)) and c;(c:(p)). 
The ending segment p;pj+1---Pn gets flipped twice by both cc; and cjc;, so 
in the end, the pattern of the last n— +1 entries will be the same in c¢;(c;(p)) 
and c;(c;(p)), because in both permutations, it will be the same pattern as it 
was in p. The middle segment p;pi+1---p; will get flipped once by both cc; 
and c;c;, so on both sides, the pattern of entries in positions 7,7+1,--- ,7 will 
be the complement of the pattern of p;pj41-- + p;- 

Finally, the set of entries in the last n — 7 +1 positions is the same in both 
ci(c;(p)) and c;(ci(p)), since both sets are equal to the vertical complement 


of the set {p;,pj+1,°-- , Pn} with respect to the set {p;,pj;41,-°- , Pn}. | 


Note that c;(c;(p)) # p, since the segment p;---p;—1 is of length at least 
two and gets flipped exactly once. Note that this is where we use the fact 
that i < j7—1. Indeed, if 1 = j — 1, then it could happen that c;c;(p) =p. An 
example for this is when p = 34251, i = 3, and 7 = 4. 


Example 1.43 
Continuing Example 1.40, we get that c3(c5(315462)) = c5(c3(315462)) = 
314562. 


Now let n > 4 be any integer. If n is even, let 


Cn = {c3, €5, C7, jen sGnait: 


If n is odd, let 
Cn = (C3, C5, C7,*** » Cn—a}. 


In both cases, C,, consists of m = |(n — 2)/2| operations. Each of these 
operations is an involution, and by Lemma 1.42, these involutions pairwise 
commute. None of these operations can be obtained as a product of the 
others, since only c; changes turns the pattern p;_p;pi+1 into something 
other than itself or its complement. Therefore, the operations in C, define a 
group H,, = Z’ that acts on the set of all permutations of length n. 

Note that with this set of generators, c;(c;(p)) # p, since the segment 
pi++*pj—1 is of length at least two and gets flipped exactly once. Also note 
that the action of the individual c; on the set of all permutations of length 
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n is independent in the following sense. If p is a permutation, and c;(p) has 
one more alternating run than p, then c;(c;(p)) also has one more alternating 
run than c;(p). Loosely speaking, the increase or decrease caused by c, de- 
pends only on the permutation p that c; is applied to, not on whether other 
transformations c; are applied to p or not. 

The action of H,, on the set of all permutations of length n creates orbits 
of size 2™. 


LEMMA 1.44 
Let A be any orbit of Hm on the set of all permutations of length n. Then 


the equality 
Se zrun(p) = z*(1 ae ae 
pEAa 


holds, where a is a nonnegative integer. 


PROOF _ Let pé A, and going through p left to right, let us apply or not 
apply each element of C, so as to minimize the number of alternating runs 
of the obtained permutation. That is, if applying c; increases the number of 
alternating run, then do not apply it, if it decreases the number of alternating 
runs, then apply it. Let qg be the obtained permutation. Then we call q the 
minimal permutation in A, since among all permutations in A, it is q that 
has the smallest number of alternating runs. Now elements of A with 7 more 
alternating runs than qg can be obtained from q by applying exactly 7 elements 
of C,, to q. As there are C2) ways to choose 7 such elements, our statement is 


proved by summing over 7. | 


Now we are ready to prove our conjecture about the high multiplicity of —1 
as a root of G,,(z). 


THEOREM 1.45 
For n > 4, the equality 


holds, where the summation is over permutations q that are minimal in their 
orbit under the action of Hm. Here m = |(n — 2)/2]. 


PROOF This follows from Lemma 1.44 by summing over all orbits. | 


So almost half of the roots of G,(z) are equal to —1; in particular, they 
are real numbers. This raises the question whether the other half are real 
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numbers as well. That question has recently been answered in the affirmative 
by Herb Wilf [330]. In his proof, he used the rather close connections between 
Eulerian polynomials, and the generating functions G,,(z) = 0,5, G(n,k)z*. 
This connection, established in [131], and given in a more concise form in 
[230], can be described by 


G2) = (H2)" ater (=). (1.18) 


2 l+w 
1—z 
1+z°* 
currence relations for A,,(z) and G,(z) to get a differential equation satisfied 
by certain generating functions in two variables. The details can be found in 
[131], pages 157-162. 


where w = The proof of (1.18) uses the similarities between the re- 


THEOREM 1.46 
(H. Wilf [330].) For any fixed n, the polynomial G,,(z) has real roots only. 


PROOF From (1.18) it follows that G,(z) can vanish only if either 
z=-—lor z= 2y/(1+ 7), where y is a zero of A,. Indeed, if y is a root of 
An(z) and y = >, then w = ae Therefore, 


1l+w? 
jl—-z 1-y 
l+z2 1+y 


Squaring both sides and solving for z, we get our claim. As we know that the 
roots of A, are real, our statement is proved. 


1.3 Alternating Subsequences 
1.3.1 Definitions and a Recurrence Relation 


The concept of alternating subsequences in permutations was introduced by 
Richard Stanley in [299]. In this section, we describe some of the major results 
of this recently developed subject, and we explain the very close connection 
between alternating subsequences and alternating runs. 


DEFINITION 1.47 An alternating subsequence in a permutation p = 
Pip2***Pn is a subsequence pi, Pir *** Pi, So that 


Pix > Piz < Diz > Pig <00* 
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Similarly, a reverse alternating subsequence in p is a subsequence Dj, Dj *** Dj, 
80 that 
Phi < Dig > Diz < Dig D> 


The length of the longest alternating subsequence of p is denoted by as(p). 
For instance, if p = 3416527, then 3165, 31657, and 427 are examples of 
alternating subsequences of p. 


Example 1.48 

If p = 35714268, then as(p) = 5. Indeed, 31426 is an alternating subsequence 
of p of length 5. On the other hand, no alternating subsequence of p can 
contain more than one of the first three entries of p, and no alternating sub- 
sequence can contain more than two of the last three entries of p. Therefore, 
no alternating subsequence of p can be longer than five. 


Interestingly, the parameter as(p) is much easier to handle in many aspects 
than its older and more studied brother, the length of the longest increasing 
subsequence of p. (We will study the latter in Chapter 4 and various later 
chapters.) The reason for this is that as(p) is more conducive to arguments 
using recurrences, due to the following fact. 


PROPOSITION 1.49 
Let p be an n-permutation. Then p contains an alternating subsequence of 
length as(p) that contains the entry n. 


PROOF Let us assume the contrary, that is, that there exists an n- 
permutation p so that each maximum-length alternating subsequence of p 
avoids the entry n. This means that all maximum-length alternating subse- 
quences of p start on the right of n, or end on the left of n, or “skip” n; that 
is, contain an entry on the left of n that is followed by an entry on the right 
of n. It is easy to see that each of these three kinds of subsequences can be 
transformed into an alternating subsequence of the same length that contains 
n. For instance, if s is an alternating subsequence of p so that a < 6 are two 
consecutive entries in s, and a is on the left of n and b is on the right of n, 
then we can replace 6 by n in s, and obtain the alternating subsequence s’ 
that is still of maximum length. If b < a, then we can replace a by n to get the 
maximum length alternating subsequence s” that contains n. Similarly, if t is 
a maximum length alternating subsequence that starts on the right of n, then 
the first entry of t can simply be replaced by n to create a maximum length 
alternating subsequence containing n. Finally, if u is a maximum length al- 
ternating subsequence ending on the left of n, then u has to end in an ascent; 
otherwise, n could be appended to the end of u to get a longer alternating 
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subsequence, which would be a contradiction. So u ends in an ascent, but 
then the last entry of wu can be replaced by n. 


The following definitions set the framework for our efforts to count permu- 
tations according to their alternating subsequences. 


DEFINITION 1.50 Let ag(n) be the number of permutations p of length 

n whose longest alternating subsequence is of length k. Set aj(0) = 1. Fur- 
themore, let by(n) = an a(n) be the number of permutations of length n 
with no alternating subsequences longer than k. 


Note that a;,(n) = bp(n) — bye_i(n). 
Our first theorem on the subject of alternating subsequences turns the ob- 
servation of Proposition 1.49 into an enumeration formula. 


THEOREM 1.51 
[299] Let 1<k<n+1. Then 


2r+s=k—-1 


ane) =(") Yo and) baarsalAaaln—), 
j=0 


where r and s range non-negative integers satisfying 2r+s=k-—1. 


PROOF We are going to build an (n + 1)-permutation p that has a 
longest alternating subsequence of length k. Let us place the entry n+ 1 into 
the (j + 1)st position of p; let L be the subsequence of the first 7 entries of p 
(since these are the entries on the left of p), and let R be the subsequence of 
the last n — 7 entries of p. 

It goes without saying that there are (") ways to choose the set of entries 
in L. 

Let us now focus on a longest alternating subsequence long of p = L(n+1)R 
containing n+ 1. Then long intersects L in an alternating subsequence alts 
of length 2r and R in a reverse alternating subsequence rev of length s = 
k —1-—2r. Clearly, no reverse alternating subsequence of R can be longer 
than rev. Therefore, as(rev) = s, so there are a,(n — 7) choices for R. The 
situation for L is a little bit more complicated. Indeed, alts may or may not 
be the longest alternating subsequence of L. If alts is of even length, then 
it is possible that alts can be extended by another entry within L, but then 
that entry cannot be followed by an ascent (and so by the entry n + 1) in any 
alternating subsequences. Therefore, as(alts) = 2r or as(alts) = 2r+1, and so 


the number of choices for L is ag,(j) + a2-41(j), completing the proof. | 
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Theorem 1.51 can be turned into generating function identities, which in 
turn can be turned into explicit formulae. The interested reader can find the 
details in [299]. 


THEOREM 1.52 
For all positive integers k <n we have 


normals E, cr(ytcia) 


r+2s<k 
r=k (mod2) 


As ay(n) = b(n) — bp-1(m), formulae for a;,(n) can be obtained from the 
preceding theorem. 
For small values of k, Theorem 1.52 yields simple formulae such as 


Ls bi (n) = 1, 
2. bo(n) = 2-1, 


8 ? 


5. bs(n) = San s)9"42(n?—5nt5) | 


6” —2(n—3)4" + (2n?—12n415)2” 
6. be(n) = grata sa an’ tanh 15)2" 


1.3.2 Alternating Runs and Alternating Subsequences 


The length of the longest alternating subsequence of a permutation is closely 
connected to the number of alternating runs as shown by the following propo- 
sition. 


PROPOSITION 1.53 
Letn > 2. Then ax(n) = $(G(n, k — 1) + G(n,k)). 


PROOF If an n-permutation p has 7 alternating runs and starts in a 
descent, then as(p) = i+ 1 as can be seen by considering the entries of p that 
are peaks or valleys, as well as the first and last entry of p. It follows from the 
pigeon-hole principle that p cannot contain a longer alternating subsequence. 
If p starts in an ascent, then as(p) =i by similar considerations. 

Therefore, the n-permutations p satisfying as(p) = k are precisely the n- 
permutations with k — 1 alternating runs starting in a descent and the n- 
permutations with k alternating runs starting in an ascent. 
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Proposition 1.53 implies that if T,(z) = ines, z2°8(P), then 


TZ) = s(l + z)Gn(z). 


So the polynomials T;,(z) have real roots only, and |n/2]| of their roots are 
equal to —1. 
The first few polynomials T;,(z) are shown below. 


1. Ti(z) =z, 

2. To(z) = z+ 27, 

3. T3(z) = z +327 + 22%, 

4. Ty(z) = 2+ 727 + 1123 + 524, 

5. Ts(z) = z+ 1527 + 4323 + 4524 + 162°, 

6. Ts(z) = z+ 31z? + 14829 + 26824 + 21125 + 6125, 

7. Tr(z) = z + 6327 + 480z9 + 134424 + 176725 + 111326 + 27227. 


1.3.3 Alternating Permutations 


Sometimes an entire permutation is an alternating sequence, leading to the 
following definition. 


DEFINITION 1.54 We say that the n-permutation p is alternating if 
the longest alternating subsequence of p is of length n. Similarly, we say that 
p is reverse alternating if the longest reverse alternating subsequence of p is 
of length n. 


For instance, 312 and 5241736 are alternating permutations. Clearly, p is 
alternating if and only if its complement, that is, the n-permutation whose 
ith entry is n+ 1 — pj, is reverse alternating. 

The number of alternating n-permutations is called an Euler number (not 
to be confused with the Eulerian numbers A(n,*)) and is denoted by E,. The 
reader is invited to verify that EF. = 1, E3 = 2, Ey = 5, and Es; = 16. The 
Euler numbers have a very interesting exponential generating function. This 
is the content of the next theorem. 


THEOREM 1.55 
Set Ej =1= E,. Then the equality 


n 
E(z)= a = sec z + tan z 
n>0 : 
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holds. 


PROOF _ Let L(n+1)R be an alternating or reverse alternating permuta- 
tion of length n+1. So L is the string on the left of the maximal entry, and R 
is the string on the right of the maximal entry. Then R is reverse alternating, 
and so is L”, that is, the reverse of L. 

This observation leads to the recurrence relation 


“(0 
2En+1= >> e Ex En-k; 
k=0 
for n > 1. In terms of generating functions, this is equivalent to 
2E'(z) = E*(z) +1, 


with F(0) = 1. 
We can easily solve the last displayed differential equation, and obtain the 
solution 


y(z) = tan (= + *) 
_ l+tan(z/2) 1+ tan?(z/2) 2 tan(z/2) 
~ 1l-tan(z/2) 1—tan?(z/2) 1 — tan?(z/2) 


= sec z + tan(z). 
i 


We point out that sec z is an even function and tan z is an odd function, so 
tan z is the exponential generating function for the Euler numbers with odd 
indices, and sec z is the exponential generating function for the Euler numbers 
with even indices. The numbers EF», are often called the secant numbers and 
the numbers F2,+41 are often called the tangent numbers. There are various 
other combinatorial objects that are counted by the Euler numbers. Exercises 
15 and 46 show two of them. 

Alternating permutations have many fascinating properties, and we will 
return to those in upcoming chapters. 


<< 
Exercises 
1. (-) Simplify the formula obtained for a(S’) in Lemma 1.3. 


2. Let p+1 bea prime. What can be said about A(p,k) modulo p+ 1? 
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10. 


11. 


12. 


13. 
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. (-) Find an alternative proof for the fact that A(n,k+1) = A(n,n—k). 


. (~) What is the value of A’, (1)? Here A,(z) denotes the nth Eulerian 


polynomial. 


. Prove Proposition 1.26. 


. We have n boxes numbered from 1 to n. We run an n-step experiment 


as follows. In step i, we drop one ball into a box, chosen randomly from 
boxes labeled 1 through 7. So during the entire experiment, n balls will 
be dropped. Let B(n, k) be the number of experiments in which at the 
end, k — 1 boxes are left empty. Prove that B(n, k) = A(n,k). 


. Deduce Theorem 1.8 from Theorem 1.7. 


. (-) Prove that for all positive integers k < n, the equality 


S(n,k) = S(n—1,k—1)+kS(n — 1,k) 


holds. 


. Prove that 


A(n,k) = > car(, * \sin—m) -(n—h)!. 


h=k-1 


Let p = pip2--:Pn be a permutation, and let b; be the number of in- 
dices 7 < iso that p; > p;. Find a formula for the number C(n,k) 
of arrays (b1, b2,--- ,b,) obtained this way in which exactly k different 
integers occur. Note that the permutation statistic defined as above is 
often called the Dumont statistic, and its value on the permutation p is 
denoted by dmc(p). 

Prove that 


n 


An(2) = 25 k1S(n,k)(z —1)"*. 


k=1 


(—) Prove that 
k 


S¢ A(n, k) < k”. 

i=1 
We say that 7 is a weak excedance of p = pypo--- pn if p; > i. Assuming 
Theorem 1.36, prove that the number of n-permutations with k weak 
excedances is A(n, k). 
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FIGURE 1.9 
The two decreasing non-plane 1-2 trees on vertex set [3]. 


14. Prove that for all positive integers n, the equality 


S(nt+lkt)= e @rGe 


m=k 
holds. 


15. A decreasing non-plane 1-2 tree is a rooted nonplane tree on vertex set 
[n] in which each non-leaf vertex has at most two children, and the 
label of each vertex is smaller than that of its parent. See Figure 1.9 
for the two decreasing non-plane trees on vertex set [3]. Let T,, denote 
the number of decreasing non-plane trees on vertex set [n]. Prove that 
Ty = En. 


16. Find a combinatorial proof for Corollary 1.19. 


17. Let r be a positive integer, and let us say that i € [n — 1] is an r-fall 
of the permutation p = pip2---pn if p(t) > p(i+1)+r. Let A(n,k,r) 
denote the number of all n-permutations with & — 1 such r-falls. The 
numbers A(n,k,r) are called the r-Eulerian numbers. Prove that 


A(n,k,r) = (k+r—1)A(n—1,k,r) 4+ (n+2—-—k—-r)A(n—1,k-1,7r). 


18. Let A(n,k,7r) be defined as in the previous exercise. Prove that 


AGie-Lier Se! an (" = ") ‘ a aa ‘ pa 


i=0 
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20. 


21. 


22. 


23. 


24. 


25. 


26. 


27. 


28. 


29. 
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Are the r-Eulerian numbers defined in Exercise 17 and the ¢-Takacs- 
Eulerian numbers defined in Problem Plus 3 identical? (Try to give a 
very short solution.) 


Let k be a fixed positive integer. Find the explicit form of the ordinary 
generating function F,(z) = }>5, S(n,k)z”. 


Prove that for all positive integers n, the equality 
2” = S$” S(n,m)(z)m (1.19) 
m=0 


holds. Recall that (z)m = z(z—1)---(e-m-+1). 


Let P(z) and Q(z) be two polynomials with log-concave and positive 
coefficients. Prove that the polynomial P(z)Q(z) also has log-concave 
coefficients. 


Is it true that if P(z) and Q(z) are two polynomials with unimodal 
coefficients, then P(z)Q(z) also has unimodal coefficients? 


Is it true that if P(z) and Q(z) are two polynomials with symmetric and 
unimodal coefficients, then P(z)Q(z) also has symmetric and unimodal 
coefficients? 


(-) 

(a) Find an explicit formula for G(n, 2). 

(b) Find an explicit formula for G(n, 3). 
Here G(n, k) is the number of n-permutations with k alternating runs. 


(—) Let n be a fixed positive integer. For what pairs (k,m) does there 
exist an n-permutation with k descents and m alternating runs? 


It follows from Exercise 21 that if n > 1, then 


Mae 


(-1)™m!S(n,m) = (-1)". 


3 
Il 


Find a direct combinatorial proof for this identity. 


Prove that for n sufficiently large, the chain of inequalities (k — 1)” < 
G(n,k) < k” holds, for all k > 2. 


A sequence f : N > C is called P-recursive if there exist polynomials 
Po, Pi,-++ , Pe € Q[n], with P, 4 0 so that 


Py(nt+k)f(ntk)t+ Pei(ntk—lf(n+k—1)+---+ Po(n)f(n) =0 
(1.20) 
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30. 


ol. 


32. 


33. 


34. 


35. 


36. 


37. 


38. 


for all natural numbers n. Here P-recursive stands for “polynomially 
recursive”. For instance, the function f defined by f(n) = n! is P- 
recursive as f(n + 1) — (n+ 1)f(n) = 0. Prove that for any fixed k, 
A(n,k) is a P-recursive function of n. 


Prove that for any fixed k, the function S(n, k) is a polynomially recur- 
sive function of n. 


A decreasing binary tree is a rooted binary plane tree that has vertex set 
[n] and root n, and in which each vertex has 0, 1, or 2 children, and each 
child is smaller than its parent. Prove that the number of decreasing 
binary trees is n!. 


Prove that the number of decreasing binary trees on [n] in which k — 1 
vertices have a left child is A(n, k). 


Let 2<2i1<n-1. Recall that we say that p; is a peak of the permutation 
Pp = P1p2°** Pn if p; is larger than both of its neighbors, that is pj_1 < p; 
and p; > pi4i1. Let n > 4, and let k > 0. Find a formula for the number 
Peak(n, k) of n-permutations having exactly k peaks. 


How many decreasing binary trees are there on n vertices in which ex- 
actly one vertex has two children? 


Use decreasing binary trees to prove that for any fixed n, the sequence 
of Eulerian numbers {A(n, k)}i<k<n is symmetric and unimodal. 


Which is the stronger requirement for two permutations, to have the 
same set of descents, or to have decreasing binary trees that are identical 
as unlabeled trees? 


The minmacz tree of a permutation p,p2---p, is defined as follows. Let 
p= umv where m is the leftmost of the minimum and maximum letters 
of p, u is the subword preceding m, and v is the subword following m. 
The minmaz tree T;” has m as its root. The right subtree of T;” is 
obtained by applying the definition recursively to v. Similarly, the left 
subtree of 77” is obtained by applying the definition recursively to u. 
See Figure 1.10 for an example. 


(a) Prove that if 1 < i < n—2, then there are n!/3 permutations p so 
that p; is a leaf in 1 ge 


(b) How many n-permutations p are there so that p,_1 (resp. pp) isa 
leaf in T;”? 


(+) Let p = pip2---pn be a permutation, and let x € [n]. Define the 
x-factorization of p into the set of strings uXA(x)a7y(x)u as follows. The 
string A(x) is the longest string of consecutive entries that are larger 
than x and are immediately on the left of x, and the string y(«) is the 
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FIGURE 1.10 
The minmax tree of p = 5613724. 


39. 


longest string of of consecutive entries that are larger than x and are 
immediately on the right of x. Finally, u(2) and v(x) are the leftover 
strings at the beginning and end of p. Note that each of A(x), y(x), u, 
and v can be empty. 


For instance, if p = 31478526 and x = 4, then u = 31, A(x) = 9, 
(x) = 785, and v = 26. 
The notion of André permutations proved to be useful in various areas 


of algebraic combinatorics. We say that p is an André permutation of 
the first kind if 


(a) there is no 7 so that p; > pi+1 > piz2, and 

(b) y(x) =0 => A(z) = 9, and 

(c) if y(a) and A(x) are both nonempty, then max A(x) < max y(2). 
Prove that p is an André permutation of the first kind if and only if 


all non-leaf nodes of the minmax tree T7” are chosen because they are 
minimum (and not maximum) nodes. 


Attach labels to the edges of a k x (n —k +1) square grid of points as 
shown in Figure 1.11. That is, both the edges of column 7 and row 7 get 
label 7. 


Take a northeastern lattice path s from the southwest corner to the 
northeast corner of the grid. This path s will consist of n — 1 steps. 
Define the weight P, of s as the product of the labels of all edges of s. 


Prove that 
A(n, k) = S- Ps, 
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1 2 3 4 5 
3 3 3 3 

1 2 3 4 5 
2 2 2 2 

1 2 3 4 5 
1 1 1 1 


FIGURE 1.11 
The labeled grid for k = 4 andn=8. 


40. 


41. 


42. 


43. 


where the sum is taken over all Ce northeastern lattice paths s from 


the southwest corner to the northeast corner. 


Prove, preferably by a combinatorial argument, that if k < (n — 1)/2, 
then the inequality G(n, k) < G(n,k + 1) holds. 


Let r be a positive integer, and modify the labeling of the vertical edges 
in the previous exercise so that the label of the edges in column 7 is 
i+r-—1 instead of i. Prove that 


A(n,k,r) =r! oS Px 


where A(n,k,7r) is an r-Eulerian number as defined in Exercise 17, the 
weight P, of a path s is still the product of the labels of its edges, and 


the sum is taken on all (771) northeastern lattice paths from (0,0) to 
(k-l1,n—r—k). 


(—) Let Q,, be the set of permutations of the multiset {1,1,2,2,---,n,n} 
in which for all 2, all entries between the two occurrences of 7 are larger 
than 7. For instance, Q2 has three elements, namely 1122, 1221, and 
2211. Note that the elements of Q,, are called Stirling permutations of 
length 2n. Prove that for n > 2, the set Q, has (2n — 1)!! elements. 


Let p = pip2-+: Pen € Qn be a Stirling permutation as defined in the 
previous exercise. Let us say that i € {1,--- ,2n} is a descent of p if 
a; > @i41 or 7 = 2n. Let us say that i € {1,2,--- ,2n —1} is a plateau 
of p if ay = Aj+1- 


(a) Let C,,; be the number of elements of Q,, with i descents. Prove 
that then for all positive integers n,i > 2, the recurrence relation 
Chi = ICn—1,i + (2n = t)Cn—-1,i-1 holds. 

(b) Let cp; be the number of elements of Q, with 7 plateaux. Prove 
that then for all positive integers n,i > 2, the recurrence relation 
Cri = 1Cn—1,% + (2n = tCn=Vi=1 holds. 


42 


44. 


45. 


46. 
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(c) Conclude that Cr; = Cn, for all positive integers 7 and n, with 
<n. 


(+) Keep the notation of the previous exercise, and let 
n 
Cr(z) = S- Cae 
i=1 


Prove that for all positive integers n, the roots of the polynomial C,,(z) 
are all real, distinct, and non-positive. 


(—) Find a direct combinatorial proof (no generating functions, no al- 
ternating runs) for the fact that for n > 2, the number as(p) is even for 
exactly half of all n-permutations p. 


A permutation p = pyp2---pn is called a simsun permutation if there 
exists no k < n so that removing the entries larger than k from p, the 
remaining permutation has two descents in consecutive positions. For 
instance, p = 35241 is not simsun, since selecting k = 3 leads to the 
substring 321. Prove that the number of simsun permutations of length 
n is Ent. 


( 


Problems Plus 


1. 


2. 


A simplicial complez is a collection A of subsets of a given set with the 
property that if EF ¢ A, and FC E, then F' € A. The sets that belong 
to the collection A are called the faces of A. If S € A has 7 elements, 
then we call S an (i — 1)-dimensional face. The dimension of A is, by 
definition, the dimension of its maximal faces. 


Prove that there exists a simplicial complex A whose set of (i — 1)- 
dimensional faces is in natural bijection with the set of n-permutations 
having exactly 7 — 1 descents. 


(a) Let T be a rooted tree with root 0 and non-root vertex set [nl]. 
Define a vertex of T to be a descent if it is greater than at least 
one of its children. Prove that the number of forests of rooted trees 
on a given vertex set with 1+ 1 leaves and j descents is the same 
as the number of forests of rooted trees with 7 + 1 leaves and 7 
descents. 


(b) Why is the above notion of descents a generalization of the notion 
of descents in permutations? 
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Rez) 


FIGURE 1.12 
The regions R,, for n = 2. 


3. Define the ¢-Stirling numbers of the second kind by the recurrence 
S(n+1,k,0) = S(n,k—-1,2)4+ k1S(n,k, 2), 


and the initial conditions $(0,0,¢) = 1, S(n,0,2) = 0 for n > 1, and 
S(0,k,) = 0 for k > 1. Note that for @ = 1, these are just the Stirling 
numbers of the second kind as shown in Exercise 8. Define the ¢-Takacs- 
Eulerian numbers by 


Ax(n,k, 2) = S- (-1y "4 Gi 1) Sern =n Olin 


r=k-1 


Note that in the special case of ¢ = 1, we get the Eulerian numbers, 
as shown in Exercise 9. Prove that these numbers generalize Eulerian 
numbers in the following sense. Modify the experiment of Exercise 6 
so that in each step, @ balls are distributed, independently from each 
other. Prove that Ark) is the probability that after n steps, exactly 
k boxes remain empty. 


4. Let k < n be fixed positive integers. Compute the volume of the region 
Ry~ of the hypercube [0,1]” contained between the two hyperplanes 
ye Vi = k-— Land Oy, a =k. See Figure 1.12 for an illustration. 


5. (a) Let p= pip2---py be a permutation, and define 


d= SY. |i-5]—|pi— pill 
1<i<j<n 
Prove that the smallest possible positive value of 6, is 2n — 4. 
(b) Which graph theoretical problem contains part (a) as a special 
case? 


6. Let G be a graph. A k-coloring of G is the number of ways to color the 
vertices of G using only the colors 1,2,--- ,k so that adjacent vertices 
have different colors. Let P(n) be the number of n-colorings of G. It 
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is then well-known that P(n) is a polynomial function of n, called the 
chromatic polynomial of G. 


Now let 
Fo(z) = 0 P(n)2”. 
n>0 
It is proved in [244] that Fe(x) = we, where Q(z) is a polynomial 
of degree m, and with nonnegative integer coefficients. 
So we can set Q(z) = Sc", wiz’, where k is the smallest number for 
which G has a k-coloring, called the chromatic number of G. 
(a) Find a combinatorial interpretation for the numbers w, in terms 
of permutations. 
(b) Explain why the polynomial Q(z) is a generalization of the Eulerian 
polynomials. 


Let n, 7, and 7 be fixed positive integers, and set 
Stijn) = S> kinky’. 
O<k<n 


Prove that 


S(é,0,n) = % i :) rlS(i,r). 


r=0 


. Let us say that a permutation p contains a very tight ascending run of 


length k if it has k consecutive entries pjpi41--+Pi+r—1 So that py; = 
pit+j—1 for 0 < 7 < k—1. In other words, the sequence p;pi+41 +++ Pi+k—1 
is a sequence of consecutive integers. 


Find a formula for the number of permutations of length r+k containing 
a tight ascending run of length at least k, if k > r. 


. The Bessel number B(n, k) is defined as the number of partitions of [n] 


into k nonempty blocks of size at most two. Prove that for any fixed n, 
the sequence B(n, 1), B(n,2),--+ , B(n,n) is unimodal. 


(a) Prove that if n = 2™ — 1 for some positive integer m, then all 
Eulerian numbers A(n,k) with 1 < k <n are odd. 
(b) Generalize the statement of part (a). 
Let A(n,k); denote the number of n-permutations with k — 1 descents 
that begin with 7. 
Prove that 
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12. 


13. 


14. 


16. 


17. 


18. 


John selects an n-permutation p at random. Jane must predict the 
descent set of p. What is Jane’s best bet? 


Prove that the Euler numbers E,, satisfy the formula 


E D) n+1 1 
sce (ee —1)krt)) 
n! (2) 2 ) (2k + 1)r+t 


Let P be a finite partially ordered set whose vertices are bijectively 
labeled with the elements of [n]. The Jordan—Hélder set of P, denoted 
by L(P), is the set of linear extensions of P. That is, L(P) is the set 
of permutations of p = pip2-:-pn so that if p; <p pj, then 7 < 7 as 
integers. 


Let W(P, z) = Vnex(P) 24), The Stanley—Neggers conjecture claimed 
that for all finite posets P, the polynomial W(P, z) has real roots only. 


(a) Let us call a finite poset P a forest if each element of P is covered 
by at most one other element. That is, P is a forest if for each 
x € P, there is at most one y € P so that x < y but there is no u 
so that x < u < y. Prove that the Stanley—Neggers conjecture is 
true for forests. 


(b) Deduce that the Eulerian polynomials A,,(z) have real roots only. 


(a) Determine the polynomial W(P, z) if P is the disjoint union of an 
m-element chain and an n-element chain. 


(b) Let Prin be the poset that consists of the disjoint union of an 
m-element chain and an n-element chain so that the elements of 
the first chain are labeled 1,2,--- ,m from the bottom up, and the 
elements of the second chain are labeled m+1,m-+2,---,m+n 
from the bottom up, with the extra relation m+1 < m added. See 
Figure 1.13 for an illustration. 

Compute W(Pimn, Z): 


(c) Prove that the Stanley—Neggers conjeture is false. 


Let a, be the number of (n + 1)-permutations that do not contain a 
very tight ascending run of length two. (See Problem Plus 8 for the 
definition of a very tight ascending run.) Find a closed formula for the 
exponential generating function A(z) = 7,59 n=. 


In the proof of Theorem 1.27, we discussed the case when neither the 
path P nor the path Q changes directions at their first common point 
X. Complete the proof for the other cases. 


Stirling permutations are defined in Exercise 42 and are the subject 
of several exercises after that. Let p = pip2---pon be a permutation 
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The poset P35. 


19. 


20. 


of the multiset {1,1,2,2,---,n,n}. We say that p is a quasi-Stirling 
permutation if there do not exist indices i << 7<k < €so that p; = pr 
and p; = pe. 
(a) Find a formula for the number of quasi-Stirling permutations of 
length 2n. 


(b) Find a formula for the number of quasi-Stirling permutations of 
length 2n that have a maximum number of descents. 


Prove that there is a unique way to write the Eulerian polynomials in 
the form 
[(n+1)/2] 
An(z) = se An perl pele et, 
k=1 


Find a combinatorial interpretation for the numbers 7y,,,, in the previous 
Problem Plus. 


Solutions to Problems Plus 


1. This result is due to Vesselin Gasharov [189], who used the same lat- 


tice path model in his solution as he used to injectively prove that the 
Eulerian polynomials have log-concave coefficients. 
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2: 


3. 


(a) This result is due to Ira Gessel [195]. Let d(f’) be the number of 
descents of a forest F' and let ¢(F') be the number of leaves of F’. 
Then let u,(a, 8) be the bivariate generating function 


un(a, 8) = So al) g1, 
F 


where the sum is over all rooted forests on [n]. Then Gessel shows 
that the trivariate generating function 


PAls 
U(z,a, B) = S- Un(a, B)— 
n! 
n>1 
is symmetric in a and £ by proving that it satisfies the functional 
equation 


14+U =(1+ aV)(14+ BU)e*-9- 8-280), 


(b) If the number of leaves is one, then the tree consists of one line, 
and the sequence of the vertices corresponds to an n-permutation. 
The notion of descents of the tree then simplifies to that of descent 
in this permutation. 


This result is due to Lajos Takacs [308], though note that his paper 
denoted the Eulerian number A(n,k) by A(n,& — 1). The main idea of 
the proof is the following. Let 


B,(n) = 3 (*) Pont) 


where P(n,k) is the probability that at the end of the trials there are k 
empty boxes. Then it can be proved that 


Bin) = shun (fan) 


by showing that both sides satisfy the same recurrence relations. Then, 
by the formula P(k,n) = >"_,(—1)’"*(7) B,(n), our claim follows. 


. The volume of Ry, is equal to A(n,k)/k!. A nice combinatorial proof 


was given by Richard Stanley [291]; though the result was probably 
known by Laplace. The main element of Stanley’s proof is the following 
measure-preserving map. It is straightforward that A(n,k)/k! is the 
volume of the set S,,,, of all points (a1, %2,--+ ,@n) € [0,1]” for which 
Xi-1 < x; for exactly k values of i. (This includes 7 = 0, where we set 
xo = 0.) Let f(a1,22,-+-+ ,&n) = (yi, Y2,°°* 5 Yn) where 


pa = OE, SO, 


Yi = 
1l4+aj_1 — 2; if x1 < 4; 
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Note that f is not defined on the set of points where x;_; = 2; for 
some i, but that is not a problem as the set of those points has volume 
zero. Apart from that, however, f maps the rest of S,, into R(n, k) 
ask—1< SO, =k-—2, <k. Stanley then shows that apart from 
a subset of volume zero of R(n,k), the map f has an inverse, and that 


f is an affine transformation of determinant (—1) 


n 


, implying that f is 


order-preserving. 


5. (a) This result is due to W. Aitken [1]. He called 6, the total relative 


(b 


) 


displacement of p. 


Let G be a graph with n vertices, and let d(x,y) be the graph- 
theoretical distance (number of edges in the shortest path) between 
x and y. Then, for a permutation p of the vertices of G, one can 
define 

dap = > > |d(pe,py) — d(x, y)). 


1<i<j<n 


Then part (a) corresponds to the special case when G is the path 
12---n. Also note that 4, of part (a) is equal to 0 if and only if 
p = 123---n or p = n---321, which is also a special case of the 
general fact that dg,» = 0 if and only if p is an automorphism of 
G. 


6. This result is due to I. Tomescu [311]. In that paper, various formulae 
are proved for the numbers wr. 


(a) Let I be an acyclic orientation of G, and let G have m vertices. 


(b 


) 


The transitive closure I’ is then a partial ordering of [m]. Let f be 
a bijective coloring of the vertices of I that is compatible with I’. 
In other words, if a < y, then f(x) <p f(y). Finally, let T(Z) be 
the set of all total orders that extend I’. In other words, the T(J) 
are all the possible choices for the bijective coloring f. 

Now for any f € T(J), note that f in fact defines a permutation 
of [m]. Let U(Z) be the set of all these permutations. Finally, let 
M(G) be the multiset obtained by taking the union of all T(J), for 
all acyclic orientations I of G, preserving the multiplicities. 

It is then proved in [311] that for any graph G, the coefficient w, 
is the number of permutations in M(G) that have & ascents. 


If G is the empty graph on m vertices, then M(G) contains all m! 
permutations of length m, and then Q(z) = Am(z)/z. 


7. There are several papers that are devoted to exploring connections be- 
tween powers of integers and Stirling numbers of the second kind, or 
Eulerian numbers. See, for instance, [203] for this result. 


8. It is proved in [219] that this number is r!(r? + r + 1). 
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9. 


10. 


11. 
12. 


13. 


It is easy to prove that 


n! 
B k SS 
(,) = Seo lk ony 
and then the result follows by checking that the sequence B(n + 1, k)/ 
B(n,k) is decreasing, therefore if it dips below 1, it has to stay below 
1. This result was published in [115]. 


(a) It is well known (see, for example, [74], Exercise 14 of Chapter 4) 
that if t is a power of 2, then ta is even, except when k = t or 
k = 1. In our case, this means that oar is always even, except in 
those special cases, and the claim follows from Theorem 1.11. 


(b) Similarly, if nm = p™ — 1 where p is a prime, then all Eulerian 
numbers A(n,k) are congruent to 1 modulo p. 


A combinatorial proof of these facts not using Theorem 1.11 can be 
found in [309]. 


This result is due to Mark Conger, and was published in [124]. 


Jane’s best bet is that p will be alternating or reverse alternating, that 
is, that D(p) = {1,3,---}A[n—1] or D(p) = {2,4,---}A[n—1]. This is 
a classic result that has several proofs. Richard Stanley’s survey paper 
[300] gives a modern proof using non-commuting variables, and mentions 
earlier proofs. 


Note that the answer is what one would intuitively expect. If 1 € D(p), 
then one would expect that 7+ 1 is more likely to be an ascent than a 
descent, and if it is an ascent, then 7 + 2 is more likely to be a descent, 
and so on. 


A comprehensive treatment of the analytical tools needed to prove this 
can be found in Analytic Combinatorics by Philippe Flajolet and Robert 
Sedgewick [169]. In particular, a special case of Theorem IV.10 of that 
book states that if f(z) = S°,39 fnz” is a function all of whose singu- 
larities in the closed disc |z| < R are simple poles, these poles are at 
points a), Q2,--- in increasing order of their distance from 0, and p; is 
the residue of f at a;, then 


In = > )pj05" + O(R™). 


j21 


All singularities of }° 55 En = tan z+ secz are simple poles. They 
are at 7/2, at —37/2, at 57/2, and so on. The residue at each pole can 
be computed, and each turns out to be —2. As R goes to infinity, the 
sum on the right-hand side converges. This proves our claim. 
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(a) This result is due to David Wagner and can be found in [322]. 


(b) Let P be an antichain of n elements. Then P is a forest, so the 
result of part (a) applies. On the other hand, L(P) = S,,, since the 
restriction that if pj <p p;, then 7 < j as integers is vacuous. So 
W(P, z) = An(z)/z, and the claim is proved. 


Results in this exercise are due to Petter Brandén [92]. 


(a) We claim that in this case, W(P, z) = mule (7) (2)z*. Indeed, 
let p = pip2--: pn € L(P). It follows from the structure of P that 
all descents of p must be formed by one element from each chain. 
That is, if p; > pj+i, then p; (“the descent top”) is one of the n 
elements of [m+n] that are larger than n, and p;+1 (“the descent 
bottom” ) is one of the m elements of [m]. If p is to have 7 descents, 
then any 7-element subset of [m+n] \ [m] and any i-element subset 
of [m] can play the role of descent tops and descent bottoms in 
exactly one (increasing) order, proving the claim. 


(b) We claim that Wp,,...2 = re (7) (Z)z*. Indeed, the only 
permutation that is not in L(P»») but is in L(P) where P is as 


in part (a) is the increasing permutation. 


(c) See [92] for a proof of the result that if M is a positive integer, 
then for m and n such that min(m,n) is sufficiently large, the 
polynomial W(Pyn,z) has more than M non-real zeros. 


Note that the Stanley—Neggers conjecture is still open for naturally la- 
beled posets, that is, for posets in which if x > y, then the label of x is 
larger than the label of y. 


First, we claim that a, = nan—-1 + (n — L)an_2, with ap = a, = 1. 
Indeed, we can insert n+1 into any position of any permutation counted 
by an—1 except immediately on the right of n. Furthermore, in any 
permutation counted by an—2, we can replace any entry 7 by the string 
i (n+1) (+1). This recurrence relation leads to the generating function 
A(z) = e77/(1— z)?. 


We will describe all cases in a fairly uniform manner, in that there is 
always an earliest intersection of P and @ with the property that before 
that intersection, P was above Q, and immediately after the intersection, 
Q was above P. In the most generic case, discussed in the text, that 
intersection is just one point X, but in general, it can be a sequence of 
edges. 


So, let us assume that the two edges of P that are attached to X are 
EE, with labels a and b, while the two edges of @ that are attached to 
X are NE, with labels c and d. This implies that 


a>b, (1.21) 
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and 
c+td<i. (1.22) 


Now proceed as follows. 


(a) If we can simply swap the parts of P and Q that start at X, do so. 
That happens if and only ifa >dandb+c <i. 
Note that the image will be a pair of paths so that we can swap 
their parts starting at X and get a pair of valid paths. 


(b) If we cannot, proceed as above because b+ c > i, thena+c> i 
by (1.21). Replace the label a by i — c, and replace the label c by 
i—a. Both of these labels decrease, so they will not violate any 
constraints with the edges preceding them. Now swap the parts of 
paths that start at X. We get an EF path with labels 7 — c and 
d, which is fine, by (1.22), and a NE path with labels 7 — a and 8, 
which is fine, by (1.21). 


Note that the image of (P,Q) will be a pair of paths so that we cannot 
just swap the parts starting at X and get a pair of valid paths, because 
we would get an EF path with labels i—c and b, and that is not allowed, 
since 1 — c < b in this case. 


So in these two cases, our map ® is injective. Now we discuss the 
remaining case, that is, when b +c <1, but a < d. 


In this case, we take a look at how P and Q continue after their coin- 
ciding edges adjacent to X. Recall that before and after X, the edges 
of P are EE, with labels a and b, while the edges of Q are NE, with 
labels c and d. The paths P and Q may coincide for a while after their 
edges labeled b and d, but when they first separate, Q will have an N 
step, and P will have an E step. (This follows from how we defined X 
in this solution.) 


Let b, b1,--- , bx be the labels of edges of P on that coinciding segment, 
and let d,d,,--- ,dx be the labels of edges of Q on that coinciding seg- 
ment. 


Furthermore, let us say that bz is followed by the EF edge labeled b’, and 
dy is followed by the N edge labeled d’. 


Now we are in a position to define our map ® in the current case. 


(a) If there is an index j so that b; > dj41 and d; > b;41, then find 
the smallest such 7, and swap the parts of P and Q that start with 
bj41 and dj;,,. Importantly, this is reversible, so we can recover 
the original pair of paths from their image. 


(b) If there is no such j, that means that for all 7, we have d; > b; . 
This is easy to prove from the fact that dp) = d >a > bo = b. So 
d > 6,, and therefore, the fact that 7 = 1 does not work implies 
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that d; > b > b;. Then we can similarly prove that dz > b2, and 
so on. In this case, we swap the parts of P and Q that start with 
b! and d’. 


In this way, we get valid paths, because 
e the inequality b, +d’ < d,+d' holds, and even the right-hand side 


was small enough to not violate the constraint in Q, and 
e the inequality dy, > by > b’ also holds. 


18. (a) This number is (2n)!/(n + 1)!, as is proved in [16]. 


(b) The number of such quasi-Sterling permutations is (n+1)"~+. See 
[161] for a proof. 


19. This is a classic result of Dominique Foata and Marcel-Paul 
Schiitzenberger [172]. 


20. The number 7¥,,,, is the number of all permutations of length n that have 
k, descents and no double descents [172]. A double descent is a descent 
that is either followed by another descent, or by the last entry of the 
permutation. 
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In One Line and Anywhere. Permutations as 
Linear Orders. Inversions. 


2.1 Inversions 
2.1.1 Generating Function of Permutations by Inversions 


In Section 1.1, we looked at descents of permutations. That is, we studied 
instances in which an entry in a permutation was larger than the entry directly 
following it. A more “global” permutation statistic is that of inversions. This 
statistic will look for instances in which an entry of a permutation is smaller 
than some entry following it (not necessarily directly). 


DEFINITION 2.1 Let p= pip2-:-pn be a permutation. We say that 
(pi, pj) 18 an inversion of p ifi <j but p; > pj. 


Example 2.2 
Permutation 31524 has four inversions, namely (3,1), (3,2), (5,2), and (5, 4). 


This line of research started as early as 1901 [254]. In this section, we survey 
some of the most interesting results in this area. The number of inversions 
of p will be denoted by i(p), though some authors prefer inu(p). It is clear 
that 0 < i(p) < (a) for all n-permutations, and that the two extreme values 
are attained by permutations 12---n and n(n — 1)---1, respectively. It is 
relatively easy to find the generating function enumerating all permutations 
of length n with respect to their number of inversions. 


THEOREM 2.3 
For all positive integers n > 2, 


S- f= TLiaZ)=H +2) pete) Ghete pinto", 
pESn, 
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PROOF We prove the statement by induction on n. In fact, we prove that 

each of the n! expansion terms of the product [,,(z) corresponds to exactly 
one permutation in S,. Moreover, the expansion term z%1z%---z%-1 will 
correspond to the unique permutation in which, for each i € [n], the entry 
«+1 precedes exactly a; entries that are smaller than itself. 

If n = 2, then there are two permutations to count, p = 12 has no inver- 
sions, and p’ = 21 has one inversion. So ee zi(?) = 1 + z as claimed. 
Furthermore, p = 12 is represented by the expansion term 1, and p’ = 21 is 
represented by the expansion term z. 

Now let us assume that we know that the statement is true for n — 1, and 
prove it for n. Let p be a permutation of length n—1. Insert the entry n into 
p to get the new permutation q. If we insert n into the last position, we create 
no new inversions. If we insert n into the next-to-last position of p, we create 
one new inversion as 7 will be larger than the last element of g. In general, 
if we insert n into p so that it precedes exactly 7 entries of p, we create i new 
inversions as n will form an inversion with each entry on its right, and with 
no entry on its left. Therefore, depending on where we inserted n, the new 
permutation q has 0 or 1 or 2, etc., or n — 1 more inversions than p did. If 
p was represented by the expansion term z%!z°?---z"-?, and n is inserted 
so that it precedes 7 entries, then g is represented by the new expansion term 
zat 292... 74-274, This argument works for all p, proving that 


In(z) = (L+z+e+s +27 4-1(2) = (1+2)(1+2+27)---+24-5-4+277-4). 


Later in this chapter we will have the techniques to write Theorem 2.3 in a 
much more compact form. 

Therefore, the number b(n, k) of n-permutations with k inversions is the 
coefficient of z* in I,(z). The fact that the polynomial J,,(z) can be decom- 
posed into a product of factors enables us to prove the following result on 
these numbers. 


COROLLARY 2.4 
For any fixed n, the sequence b(n, 0), b(n, 1),--- , b(n, (oy is log-concave. 


PROOF Let us call a polynomial log-concave if its coefficients form a 
log-concave sequence. It is then not hard to prove (see Exercise 22 of Chapter 
1) that the product of log-concave polynomials is log-concave. The previous 
theorem shows that the generating function of our sequence is the product of 
several log-concave polynomials (of the form 1+ z+27+---+ 2"). Therefore, 
our sequence itself is log-concave. 


The first few values of the numbers b(n, &) are shown in Figure 2.1. 
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n=1 1 

n=2 1 1 

n=3 1 2 2 1 

n=4 tof &@ So 4 
n=5 149 15 20 22 2015941 


FIGURE 2.1 
The values of b(n, k) for n < 5. Row n starts with b(n, 0). 


We would like to point out that it is not true that the polynomial J,,(z) has 
real roots only. Indeed, if n > 3, then I[,(z) is divisible by 1+ z+ 27, and 
therefore has some complex roots. 

We have found, with not much effort, the generating function of the numbers 
b(n, k) for fixed n. An enumerative combinatorialist will certainly ask next 
whether it is possible to find a recursive, or even better, an explicit formula 
for these numbers, just as we did for the numbers A(n,k) in Section 1.1. As 
we will see, the latter is a somewhat more difficult task. 

To start, we prove a recurrence relation. 


LEMMA 2.5 
Letn>k. Then we have 


b(n +1,k) = b(n +1, k— 1) + O(n, k). (2.1) 


PROOF _ Let p = pip2---pr+i be an (n + 1)-permutation with & inver- 
sions, where k < n. If pny4i = n+ 1, then we can remove the entry n+ 1 
from the end of p and get an n-permutation with k inversions. If pj =n+1 
for 7 <n, then let us interchange n + 1 and the entry immediately following 
it. This results in an (n + 1)-permutation with k — 1 inversions in which the 
entry n+1 is not in the first position. However, all (n+1)-permutations with 
k — 1 inversions have that property (that n+ 1 is not in the first position) 
as putting n+ 1 to the first position would result in at leastn >k >k-1 
inversions. This completes the proof, and also shows why the condition n > k 
is needed. 


Even without going into details, it is obvious that (2.1) does not hold in 
general, that is, when k > n. For instance, if k > (5) 72; then b(n + 1,k) < 
b(n +1,4—1), which makes it impossible for (2.1) to hold as b(n, k) > 0. See 
Exercise 28 for a recurrence relation for the case when k > n. 

Finding an explicit formula for the numbers b(n, k) is significantly more 
difficult, even if we assume n > k. A little examination of the polynomial 
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In(z) = (2) b(n, k)z* shows that 


wna) = (77) ("5") n>. 


In order to see how these results are obtained, and to obtain a general 
formula, we need some notions that are probably well-known to most readers. 


DEFINITION 2.6 Letn be a positive integer. If aj +ag+---+ap =n, 
and the a; are all positive integers, then we say that the k-tuple (a1, a2,--- , ax) 
is a composition of n into k parts. If the a; are all nonnegative integers, then 
we say that the k-tuple (a1,a2,--- ,@x) is a@ weak composition of n into k 
parts. 


In the unlikely event that the reader has not met compositions before, the 
reader should take a moment to prove that the number of compositions of n 
into k parts is es) whereas the number of weak compositions of n into k 
parts is (eo 

For instance, to get b(n, 2) as the coefficient of z? in I,(z), one has to count 
the weak compositions of 2 into n — 1 nonnegative parts, the first of which 
is at most 1. Indeed, one needs to find the coefficient of z? in the generating 
function [,(z) = (1+ 2)1+2+27)---(l1+2+-::+2"'). The number of 


all weak compositions of 2 into n — 1 parts is (ee: — *) = (,",) =n, one of 


which consists of a first part equal to 2. This provesdhst b(n, 2) =n—1. See 
Exercises 3 and 4 for proofs in the cases of k = 3 and k = 4. 

This line of formulae suggests that maybe the formula for b(n,k) will be 
obtained by taking the difference of two suitably chosen binomial coefficients. 


However, this conjecture is false as we have 


b(n, 5) = Cs) = ee ra 


Further conjectures claiming b(n,k) to satisfy a formula of comparable sim- 
plicity also turn out to be false. The truth is a bit more complicated than 
that. 
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FIGURE 2.2 
The Ferrers shape of p = (5, 2,1). 


Our main tool in finding the correct formula comes, remarkably, from the 
theory of integer partitions. Many readers are probably familiar with the 
following definition. 


DEFINITION 2.7 Let ay > a2 >-:: > am > 1 be integers so that 
a, tag+-+:+-+a@m =n. Then the array a = (a1,42,:++,@m) ts called a 
partition of the integer n, and the numbers a; are called the parts of the 
partition a. The number of all partitions of n is denoted by p(n). 


Partitions of the integer n are not to be confused with partitions of the set 
[n]. If there is a danger of confusion, we may refer to the objects we have just 
defined as integer partitions and to the objects we defined in Section 1.1 as 
set partitions. 


Example 2.8 
The integer 5 has seven partitions, namely (5), (4,1), (3,2), (3,1,1), (2,2, 1), 
(2,1,1,1), and (1,1,1,1,1). Therefore, p(5) = 7. 


The topic of integer partitions has been extensively researched for several 
centuries, from combinatorial, number theoretical, and analytic aspects. See 
[13] for a survey. 

We will use the following simple, but extremely useful, representation of 
partitions by diagrams. A Ferrers shape of a partition p = (a1, a@2,+-- , az) is 
a set of n square boxes with sides parallel to the coordinate axes so that in 
the ith row we have a; boxes and all rows start at the same vertical line. The 
Ferrers shape of the partition p = (5,2,1) is shown in Figure 2.2. Clearly, 
there is an obvious bijection between partitions of n and Ferrers shapes of size 
n. 

We will need some basic facts about the generating functions of various 
partitions. 


PROPOSITION 2.9 
The ordinary generating function of the sequence of the numbers p(n), where 
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Co 


S/ r(n)2” = [T ; 7 (2.2) 


n>0 i=1 


PROOF ~~ Wecan decompose the right-hand side as 


(14 z+ 2774 -\(14 42744 vee Joss (Lp 2t+ 27 4.:-), 


It is now clear that the coefficient of x” in this product is equal to the 
number of vectors (ci, ¢2,:+:) with nonnegative integer coefficients for which 
ve ics = n. Note that such a vector can have only a finite number of 
nonzero coordinates. Finally, there is a natural bijection between these vec- 
tors and the partitions of n. This bijection maps (c1, c2,--- ) into the partition 
that has c; parts equal to 7. So the coefficient of z” on the right-hand side is 


p(n). 


COROLLARY 2.10 
Let pa(n) be the number of partitions of n into distinct parts. Then the equality 


S> pa(n)z” = J Jat) @3) 
n>0 i=1 


holds. 


COROLLARY 2.11 
Let p(n,m) be the number of partitions of n into at most m parts. Then the 


equality 
. m 1 
y p(n, m)z” = | | rn (2.4) 


holds. 


A pentagonal number is a non-negative integer n satisfying n = $(3j? + j) 
for some non-negative integer 7. So the first few pentagonal numbers are 
0,1,2,5,7,---. 

The following partition identity is the most interesting one in our quest for 
the formula for the numbers b(n, k). It is not nearly as easy as the preceding 
two corollaries. 


LEMMA 2.12 
Let po,a(n) (resp. pe,a(n)) be the number of partitions of n into an odd number 
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P q 


FIGURE 2.3 
We have b(p) = 3, g(p) = 1, and 6(q) = 1, g(q) =2. 


of distinct parts (resp. an even number of distinct parts). Then 


0 ifn is not pentagonal, 
De,d(?) = Do,a(n) = 


(-1)9 ifn = 9(37? + 5). 


Example 2.13 
Let n = 6, which is not a pentagonal number. Then pe.aq(6) = 2, since the 
suitable partitions are (5,1) and (4,2). On the other hand, po,4(6) = 2, since 
the suitable partitions are (6) and (3, 2,1). 

Now let n = 7, which is a pentagonal number. Then p,,4(7) = 3, since the 
suitable partitions are (6,1), (5,2), and (4,3). On the other hand, p,,4(7) = 2, 
since the suitable partitions are (7) and (4, 2,1). 0 


PROOF We will define a map ¢ from the set of partitions of n into an 
even number of distinct parts to the set of partitions of n into an odd number 
of distinct parts. We will then show that if n is not a pentagonal number, 
then our map is a bijection, proving the first part of our theorem. The second 
part of the theorem will be proved by analyzing why ¢ is not a bijection if n 
is pentagonal. 

Let q = (q1,92,°:- ,q;) be any partition of n. We define two parameters 
for g. The simpler one, g(q) (for gray) is just the size of the smallest part of 
q. In other words, g(q) = q;. The other one, b(q) (for black) is the length of 
the longest strictly decreasing subsequence (q1, q2,---) of parts of g in which 
each part is exactly one less than the part immediately preceding it. Note 
that by definition, this strictly decreasing subsequence must start with q,. So 
in particular, if gg ~ qi — 1, then b(q) = 1. 

See Figure 2.3 for an illustration. 

The set of black boxes will be called the outer rim of a Ferrers shape. 

Now let p be a partition of n into an even number of distinct parts. We 
distinguish two cases. 


1. If b(qg) > g(q), then we remove the last part of g, and add one to each 
of the first g(q) parts of q, to get the partition ¢(q). This operation 
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decreases the number of parts by one, and always keeps all parts distinct. 
The procedure can always be carried out except in the case when each 
part of q, including the last one, is one less than the preceding one, and 
b(q) = g(q), such as in the case of q = (3,2) or g = (7,6,5,4). Indeed, 
it is precisely in that case that we have no row left into which we could 
place the last gray box. We do not define ¢ in that exceptional case. 


Note that at the end of this procedure, b(¢(q)) < g(¢(q)) holds. This is 
because b(#(q)) = g(q), and g(¢(q)) is at least as large as the next-to-last 
row of q. 


2. If b(¢) < g(q), then we decrease each of the first b(q) parts of q by one, 
then affix a last part of size b(q) to the end of q to get the partition 
¢(q). This operation increases the number of parts by one, and keeps 
the parts all distinct, except in the case when each part of q, including 
the last one, is one less than the preceding one, and b(¢) = g(q) — 1, 
such as in the case of q = (4,3). Indeed, it is precisely in that case that 
the new last part that we create is just as long as the part immediately 
preceding it, that is, of length g(q) — 1. We do not define ¢ in that 
exceptional case. 


Note that at the end of this procedure, b(¢(q)) => g(¢(q)) holds. Indeed, 
the procedure does not decrease b and does not increase g. 


So the only times when ¢(q) is not defined are as follows. 


(A) When b(q) = g(q), and the last part of g and the outer rim of the Ferrers 
shape of q both consist of 7 boxes. Therefore, the partition q is of the 
form q = (273 — 1,27 —2,--- ,7), son= Gri, 


(B) When 6(g) = g(q)—1, then the outer rim of g consists of 7 boxes, and the 
last part of gq is 7 +1. Therefore, in this case, gq = (27,27-—-1,--- ,j +1), 
son = Gis 

Note that in both of these exceptional cases, the integer 7 has to be even, 
to assure that q has an even number of parts. 

So in all cases but the exceptional cases (A) and (B), our function ¢ maps 
into the set of partitions of n into an odd number of distinct parts. It is clear 
that ¢ is one-to-one since it is one-to-one on both parts of its domain, and 
the images of the two parts are disjoint, as explained in the last sentence of 
the definition of ¢(q) in each case. Let us examine whether ¢ is surjective. 

Let r be a partition of n into an odd number of distinct parts, and let us 
try to find the preimage of r under ¢. If g(r) < b(r), then this preimage 
can be found by removing the last part of r and adding one to each of the 
first g(r) parts of r. This can always be done, unless g(r) = b(r), and each 
part of r, including the last one, is one less than the preceding one, as in 
q = (5,4,3). If g(r) > b(r), then this preimage can be found by decreasing 
each of the first b(7) parts of r by one, and creating a new, last part of r that 
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is of size b(r). This procedure does not yield a partition into distinct parts 
only if b(r) = g(r) — 1, and each part of r is one less than the preceding one, 
as in q = (6,5,4). So we can find the unique preimage ¢~!(r) of r unless 


A’) b(r ,so r= (27 —1,27-—2,--- ,7), and therefore, n = BIRDS or 

( = g(r j ji ys a 
gs _ (3j4Di 

(B’) b(r) = g(r) -1, sor = (27,27 -1,--- ,7 +1), and therefore, n = “4. 


Also note that in cases (A’) and (B’), the number j has to be odd to ensure 
that r has an odd number of parts. 

In other words, if n is not a pentagonal number, then ¢ is a bijection 
from the set of partitions enumerated by pe,q(n) onto the set of partitions 
enumerated by po,a(n). 

In exceptional cases (A) and (B), (which occur when n = Gris for some 
even positive integer j), there is one partition in the domain of ¢ that does 
not get mapped into a partition consisting of an odd number of parts, showing 
that pe,a(n) — po,a(n) = 1 = (—1)/ as 7 is even. 

In exceptional cases (A’) and (B’), (which occur when n = GiDs for some 
odd positive integer j), there is one partition of n into an odd number of 
distinct parts that does not have a preimage under ¢, proving that pe,a(n) — 


Do,d(n) = —1 = (—1)/ as J is odd. | 


Now let pa(n,m) be the number of partitions of n into m distinct parts. 
As a consequence of the previous lemma, note that if n is not of the form 
(377 + j)/2 or (37? — j)/2 for some nonnegative integer 7, then we have 


n 


32 (-1)™paln, m) = 0. 


m=1 


Otherwise, we have 
n 


S>(-1)"pa(n,m) = (—1)5. 


m=1 
The following remarkable Corollary links the pentagonal numbers to the 
enumeration of permutations according to their number of inversions. 


COROLLARY 2.14 
[Euler’s formula.] The identity 


al ee re ae 
=S°(-1 735 ?+5)/2 
jJEZ 


holds. 
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The reader should consider for a minute how fundamentally interesting this 
result is. Even if someone has no interest in, or knowledge of, integer partitions 
and inversions in permutations, he or she could still very well ask what the 
power series form of the product [],,(1— z’) is. And the interesting answer 
is that most coefficients in that power series are 0, and the nonzero coefficients 
all have absolute value 1, and they all occur at exponents corresponding to 
pentagonal numbers. 


PROOF The left-hand side is similar to the generating function of the 
numbers pa(n) as given in (2.3), except for the negative sign within each term. 
This implies that the coefficient of z” on the left-hand side is not simply the 
sum of all the numbers pg(n,m), but their signed sum >°,,(—1)™pa(n,m). 
We know from Lemma 2.12 that this sum is 0, except when n is of the form 
(377 + j)/2 or (37? — j)/2, in which case this sum is equal to (—1)/. This 
completes the proof. 


We mention that the rather unusual summation >> jez im Euler’s formula 
is used to include pentagonal numbers of the form (377 +7) /2 and (37? — j)/2 
in the same sum. One can think of the sum DW jeg (17207 +9)/2 as the sum 
in which j ranges through all integers in order 0, —1, 1, —2, 2, —-3,3,---. For 
j € Z, let us set dj = (37? + j)/2. 

Recall that by T heorem 2.3, the polynomial J,,(z) can be rearranged as 


In(z) = ][@+2+-- ) = [4 


i=l i=l 


Let k <n. While [,,(z) is a polynomial and ou is an infinite product, 
their factors of degree at most k agree; therefore, their coefficients for terms 
of degree at most k also agree. So our task is reduced to finding the coefficient 


of z* in poe 
—n n+h— 
ita eet et 
h>0 

where we set (a) = 1. In order to get a term with coefficient k in the product 
f(z): (1-2), we have to multiply the term (—1)/z@0°+3)/2 = (—1)32% of 
f(z) by the term of (1 — z)~” that has exponent k — d;, that is, in which 
h=k-—d;. Therefore, we have proved the following theorem. 


THEOREM 2.15 
Let n >k. Then the coefficient of z* in In(z), or, equivalently, the number 
of n-permutations with k inversions, is 


b(n, k) = So(-1)" ie 7 ‘) (2.5) 


j 
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FIGURE 2.4 
The first five pentagonal numbers. 


where j is such that the pentagonal number d; is at most as large as k. 


The first few pentagonal numbers are shown in Figure 2.4. 
Expanding (2.5), we see that ifn > k, then the formula for b(n, k) starts as 


follows. 
n+k-1 n+k-2 n+k-3 
sim (eT) Cae) 
4 n+k-6 " n+k—-8 _ 
k—5 k-7 


2.1.2 Major Index 


There are other permutation statistics that are equidistributed with the num- 
ber of inversions. That is, there exist other permutation statistics stat so 
that for all non-negative integers n and k, the number of n-permutations p 
satisfying stat(p) = k is equal to b(n, k). The most famous of these statistics 
is the major index, which was named after the rank of its inventor, Percy 
MacMahon, in the British Army. 


DEFINITION 2.16 Let p = pip2:::pn be a permutation, and define 
the major index or greater index maj(p) of p to be the sum of the descents of 


p. That is, maj(p) = Vie D(p) a 


Example 2.17 
If p = 352461, then D(p) = {2,5}; therefore, maj(p)=7. UO 


In 1916, MacMahon showed [248] the following surprising theorem by prov- 
ing that the two relevant generating functions were identical. It was not until 
1968 that a bijective proof was found by Dominique Foata [170], who worked 
in a more general setup. Another proof that can be turned into a bijective 
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proof is given in Exercises 31 and 32. We present Foata’s proof in the simpli- 
fied language of permutations. 


THEOREM 2.18 
For all positive integers n and all nonnegative integers k, there are as many n- 
permutations with k inversions as there are n-permutations with major index 


k. 


In other words, the permutation statistics “number of inversions,” which we 
denoted by 7, and “major index,” which we denoted by maj, are equidistributed 
on S,. If a permutation statistic s has the same distribution on S, as 7, then 
s is called Mahonian. 


PROOF (of Theorem 2.18) For any permutation p = pip2--- pn, we call 
the entry p; large if p; > pn, and we call p; small if p; < py. 

We are going to prove our statement by recursively defining a bijection 
@: Sy, — Sy so that for all p € S,, the equality maj(p) = i(¢(p)) holds. Our 
map ¢ will have the additional feature of keeping the last element of p fixed. 

It will not surprise the reader that we define #(1) = 1 for the initial case of 
n =1, and $(12) = 12 and $(21) = 21 for the case of n = 2. 

Now let us assume that we have defined ¢ for all (n — 1)-permutations, 
and the ¢ fixes the last entry of each such permutation. In order to define ¢ 
for all n-permutations, we distinguish two cases. Let p = p,p2:--Pn be any 
n-permutation. 


1. First we consider the case when p,—; is a small entry. In this case, 
take Wp = $(p1p2--*Pn—1) = %142°**Gn—-1- Let 4i,,4i2,°** Gi; be the 
small entries of p in wp, that is, those that are less than p,. Note that 
Pn—-1 = Mn—1- Set ig = 0. Let Q; = Gij;-141°°°qi;- In other words, the 
Q,; provide the unique decomposition of w into subwords that contain 
exactly one small entry, and contain that small entry in the last position. 
For instance, if q1---qg = 425613, then there are two small entries, 1 
and 2, and therefore, Q; = 42, and Q2 = 561. Now define 


Q,; if Q; is of length at most 1, 
f(Q;) = 


Lm LQ°**Lm—1 if Qj = 41 XQ°++ Xm, with m > 2. 


Finally, define 
f (Wp) = F(Q1) f(Q2) --- F(Qx), 


and 
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Example 2.19 


Let n = 5, and p = 54213. Then we have wy = $(5421) = 5421, and 
Q = 542, Q2 =1. Therefore, f(w,) = 2541, and so $(p) = 25413. 


2. When p,,—; is a large entry, the procedure is very similar. The only dif- 
ference is in the definition of the strings Q;. In this case, the Q; provide 
the unique decomposition of w into subwords that contain exactly one 
large entry, and contain that large entry in the last position. 


Example 2.20 


Let n = 5, and let p = 13452. Then we have w, = $(1345) = 1345, 
and @) = 13, Q2 = 4, and Q3 = 5. Therefore, f(w,) = 3145, and so 
o(p) = 31452. J 


It is easy to see that @: S, — S), is a bijection. Indeed, verifying both 
cases, one sees that the first rule was used to create ¢(p) if and only if the last 
element of d(p) is larger than the first element of 6(p). Once we know which 
rule was used to create ¢(p), we can recover w, from f(w,). Indeed, if the first 
(resp. second) rule was used, then the f(Q;) are the subwords that contain 
only one small (resp. large) entry, and contain that small (resp. large) entry 
in the first position. As f is a bijection, recovering the f(Q;) this way allows 
us to recover the Q;, and therefore, w, itself. Finally, @ : Sp_1 + Sn_1 is a 
bijection by induction, so we recover p1p2--+Pn—1 from f(pip2--:Pn—1) = Wp. 

We still need to prove that ¢: S;, + S, has the desired property, that is, it 
maps a permutation with major index & into a permutation with k inversions. 
We accomplish this by considering the two above cases separately. 


1. When p,_1 is a small entry, then 


maj(p) = maj(pip2-+*Pn—1) = 1(b(pip2-**Pn—-1)) =t(wp). (2.6) 


How does the map f change the number of inversions of w(p)? It does 
not change the order among the small entries, or among the large entries. 
If a small entry belongs to the subword Q; of length t > 1, then it jumps 
forward and passes all t — 1 large entries of Q;, decreasing the number 
of inversions by t — 1. 


As each large entry will be passed by one small entry, the total decrease 
in inversions is equal to the number of large entries, that is, to n — py. 
However, affixing p, to the end of f(w,) will create precisely n— pn new 
inversions. Therefore, 


(PP) = UF (Wp)Pn) = twp), 
which, compared to (2.6), shows that maj(p) = 1(¢(p)) as claimed. 
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2. When pp_1 is a large entry, then 


maj(p) = maj(pip2 ++: Pn—1) + (n— 1) (2.7) 

= i(O(pipe-+*Pn—-1)) t+n-1l=i(wp)tn—-1. (2.8) 

When f is applied to wy, each large entry belonging to a subword of 

length t > 1 jumps forward, passes all t— 1 small entries of its subword, 

and increases the number of inversions by tf — 1. Each small entry is 

passed by one large entry, so the total increase in the number of inver- 

sions is equal to the number of small entries, that is, p, — 1. On the 

other hand, affixing p, to the end of f(w,) will create precisely n — py 
new inversions. Therefore, 


i(O(D)) = if (wp )Pn) = t(wp) + (Pn -1) +(n—pn) = i(wp)+n—-1, (2.9) 
which, compared to (2.7), shows that again, maj(p) = 7(¢(p)) as claimed. 


Other examples of Mahonian statistics can be found among the exercises. 


2.1.3 Application: Determinants and Graphs 
2.1.3.1 Explicit Definition of Determinants 


There are several undergraduate mathematics courses and textbooks that 
only give a recursive definition of the determinant of a square matrix. That 
is, det c d is defined to be equal to ad — bc, and then the determinant of 


the n x n matrix A = (a;;) is defined to be 
det A = So (-1)) 4a Ary (2.10) 
j=l 


where Aj, is the (n —1) x (n—1) matrix obtained from A by removing the 
first row and the jth column. 

If that is the only definition of determinants the reader has seen, he may 
find the following result interesting. 


THEOREM 2.21 
Let A= (aij) be ann x n matric. Then we have 


det A= S° (-1)' arp, aap. +++ Onp,- (2.11) 
PESn 


That is, det A is obtained by taking all n! possible n-tuples of entries so that 
there is exactly one of the n entries in each row and each column, multiplying 
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the elements of each such n-tuple together, finally taking a signed sum of 
these n! products, where the sign is determined by the parity of i(p), and p is 
the permutation determined by each chosen n-tuple. 

In other words, the n-tuples correspond to all possible placements of n rooks 
on an n x n chessboard so that no two of them hit each other. 


Example 2.22 

Let n = 3. Then there are three 3-permutations with an even number of 
inversions, namely 123, 312, and 231, and there are three 3-permutations 
with an odd number of inversions, namely 132, 213, and 321. Therefore, we 
have 


det A = Q11422033 + €13421 032 + €12023431 — 411023432 — 412021433 — 413422431. 


U 


PROOF (of Theorem 2.21). We prove the statement by induction on n, 
the initial cases of n = 1 and n = 2 being obvious. Let us assume that the 
statement is true for (n — 1) x (n — 1) matrices. That means that 


det Ary = S>(-1)" a2q, 4395 *** Ongns (2.12) 
q 


where ¢ = q293°::@n is a partial permutation, that is, a list of the integers 
1,2,---,7-1,7 +1,--- ,n in some order. 

Therefore, a1; det Ay; will contribute the products of all n-tuples starting 
with a1; to det A. In other words, a1; det Ai; will correspond to all non-hitting 
rook placements in which there is a rook in position 7 of the first row. This 
argument can be applied for each 7. So our theorem will be proved if we can 
show that the signs of these products are what they should be. 

Substituting the expression provided for Aj; by formula (2.12) into for- 
mula (2.10), we see that the sign of the n-tuple that belongs to q becomes 
(—1)9-1+*@ , And indeed, the permutation p = jq2q3--- dn has precisely j —1 
more inversions than the partial permutation q = q2q3°-:dn as j is larger 
than 7 — 1 other elements. This shows that the contribution of this n-tuple is 
indeed counted with sign (—1)4~1. 


2.1.3.2 Perfect Matchings in Bipartite Graphs 


The explicit definition of the determinant has some surprising applications in 
graph theory. A simple graph is a graph with no loops or multiple edges. A 
perfect matching M in the simple graph G is a set of pairwise vertex-disjoint 
edges covering all vertices of G. In other words, each vertex of G belongs 
to exactly one edge in M. The graph G is called bipartite if the vertex set 
of G can be cut into two parts X and Y so that all edges of G have one 
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vertex in X and one vertex in Y. The truncated adjacency matriz of a simple 
bipartite graph G is the matrix B(G) = (6;;) in which 6,;; = 1 if there is an 
edge between i € X and j € Y, and bj; = 0 otherwise. Note that the rows of 
B represent the vertices of X, and the columns of B represent the vertices of 
Y.. 

Whether a bipartite graph has a perfect matching is an interesting and 
well-studied question. A sufficient and necessary condition for this existence 
problem is the well-known Marriage Theorem, which is included in most ele- 
mentary graph theory books, such as [74]. 

The concept of truncated adjacency matrices provides us with a sufficient 
condition for the existence of a perfect matching that is very easy to verify. 


THEOREM 2.23 
Let G be a simple bipartite graph with |X| = |Y| = n that does not have a 
perfect matching. Then det B(G) = 0. 


In other words, if det B(G) 4 0, then G has a perfect matching. 


PROOF ~ We prove that det B(G) = 0 by showing that all n! summands 
in the explicit definition (2.11) of B(G) are equal to 0. This is because the 
existence of a nonzero term bp, b2p, °- + bnp, would be equivalent to the exis- 
tence of a perfect matching, namely the perfect matching in which 7 € X is 
matched to p; € Y. 


We also note that the number of all perfect matchings of G can be obtained 
by computing the permanent of B(G) that is defined by 


per B(G) = S> dip, bap. +++ Onp, 
pESn 


That is, per B(G) is defined just like det B(G), except that each term is added 
with a positive sign. 


2.2 Inversions in Permutations of Multisets 


Instead of permuting the elements of our favorite set, [n], in this section 
we are going to permute elements of multisets. We will use the notation 
{1% ,2%,..- ,k**} for the multiset consisting of a; copies of 2, for all i € [k]. 

For our purposes, a permutation of a multiset is just a way of listing all its 
elements. It is straightforward to see, and is proved in most undergraduate 
textbooks on enumerative combinatorics, that the number of all permutations 
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of the multiset K = {1%,2%,--- ,k%} is 


n! 


ay!ag!:+- an!’ 


where n = a1 +02 +--- +4. 

An inversion of a permutation p = p,p2--:Pn of a multiset is defined simi- 
larly to the way in which it was for permutations of sets, that is, (7,7) is an 
inversion if 1 < j, but p; > pj. 


Example 2.24 
The multiset-permutation 1322 has two inversions: (2,3) and (2,4). 


If we want to generalize Theorem 2.3 for permutations of multisets, that is, 
if we want to count permutations of multisets according to their inversions, we 
encounter exciting and surprising connections between the objects at hand, 
and a plethora of remote-looking areas of combinatorics. 

Our goal is to find a closed expression for the sum 


ye, (2.13) 


pEesrK 


where Sx denotes the set of all permutations of the multiset K. We cannot 
reasonably expect something quite as simple as the result of Theorem 2.3 as 
the formula to be found will certainly depend on each of the a;, and not just 
their sum n. Therefore, the reader will hopefully understand that we need 
some new notions before we can find the desired closed formula for (2.13). 

Let [pn] = 1+¢q+q?+---+q""!, the polynomial whose importance we 
know from Theorem 2.3, and let [n]! = [1] -[2]-----[n]. Do not confuse 
[n] = {1,2,--+ ,n}, which is a set, and [n] = 1+q+q?+-:-+¢""", which isa 
polynomial. One way to avoid the danger of confusion is to use the notation 
[n]q instead of [n], but that notation can result in crowded formulas. Note 
that if we substitute q = 1, then [i] = 7, and therefore [n]! = n!, so this concept 
generalizes the concept of factorials. The crucial definition of this section is 
the following. 


DEFINITION 2.25 Let k and n be positive integers so thatk <n. Then 
the (n, k)-Gaussian coefficient or g-binomial coefficient is denoted by [7] , and 
is given by 


Note that [7] = |,7. 
reduces this definition to that of the usual binomial coefficients. This, and 


clearly holds. Also note that substituting g = 1 
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other connections between binomial and g-binomial coefficients, will be further 
explored shortly. Finally, we can define g-multinomial coefficients accordingly. 


DEFINITION 2.26 — Let a1, d2,:-- ,ax be positive integers satisfying 


a a; =n. Then the (a1, a2,--- , a, )-Gaussian coefficient, or g-multinomial 
coefficient is denoted by Pa Pine [3 and is given by 

| n [n]! 

a1,82,°"* , aK [az]![ag]!--- [ax]! 


We point out that similarly to multinomial coefficients, the g-multinomial 
coefficients satisfy the identity 


cfewle iro ee ee se 


Note that using this terminology, Theorem 2.3 can be written as 


Ss) 7 = [all 


PESn 


The Gaussian coefficients look like rational functions of g, but, as we will 
soon see, it is not difficult to prove that they are in fact polynomials in q. 
Even more strongly, they are polynomials with positive integer coefficients. 
This is why sometimes they are called Gaussian polynomials. 


2.2.1 Application: Gaussian Polynomials and Subset Sums 


Before we start applying Gaussian polynomials to obtain generating functions 
of multiset permutations, it seems beneficial to take a look at one of their 
several natural occurrences. The advantage of this will be that the reader 
will see in what sense the Gaussian coefficients Ea are generalizations of the 
binomial coefficients Gy: That, in turn, will be helpful in putting into context 


the recurrence relations of Gaussian coefficients that we are going to use. 


THEOREM 2.27 

Let n and k be fixed non-negative integers so thatk <n. Let a; denote the 
number of k-element subsets of [n] whose elements have sum i+ (eri that 
is, 4 larger than the minimum. Then 


k(n—k) 


| = = aig’. (2.15) 


In other words, Bel is the ordinary generating function of the k-element 
subsets of [n] according to the sum of their elements. 
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Example 2.28 

Let n = 4 and & = 2. Then, among the six 2-element subsets of [4], two, 
namely {1,4} and {2,3}, have sum 5, and all other sums from 3 to 7 are at- 
tained by exactly one subset. Therefore, the right-hand side of (2.15) becomes 
1+q+2q? +q° +44, which is indeed equal to 

B (q* -1)(q* -9) 


9 “(alga +1) +q+1). 


PROOF (of Theorem 2.27) We prove the statement by induction on n, 
the initial case of n = 1 being obvious. Let us assume that the statement is 
true for n — 1 and prove it for n. Exercise 21 shows that 


| = qr. al x ial (2.16) 
Therefore, our induction step will be complete if we can show that the poly- 
nomials yo aq’ satisfy the same recurrence relation. That is, let b; be 
the number of k-element subsets of [n — 1] whose sum of elements is i + CS) 
and let c; be the number of (k — 1)-element subsets of [n — 1] whose sum of 


elements is 7 + (2); we then need to show that 


k(n—k) k(n—k—-1) (k—1)(n—k) 
dS ag =| DD bag + fae * SO ead’ 
i=0 i=0 i=0 


This is the same as showing that a; = b; + cjq”~* for all i, where undefined 
coefficients are to be treated as zero. However, the last equation is clearly 
true as a k-subset of [n] either does not contain n, and then it is accounted 
for by 6;, or it does, and then it is accounted for by c;_(n—%), because of the 


shift in the definition of c;. I 


2.2.2 Inversions and Gaussian Coefficients 


Now we are ready to announce and prove the result describing the generating 
function of multiset-permutations according to the number of their inversions. 


THEOREM 2.29 
Let K = {1% ,2%,--- ,k**} be a multiset so that = a; =n, and let Sx 
denote the set of all permutations of kK. Then the equality 


S- gi?) = | = | (2.17) 
A1,42,°°° , ak 


pEesrK 
holds. 
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PROOF First we prove the statement in the special case of k = 2. In 
this case, we will write K’ instead of K, to remind the reader that K’ is a 
multiset consisting of a; copies of 1 and ag copies of 2, where that aj +a2 =n, 
and an inversion is an occurrence of a 2 on the left of a 1. We need to prove 


that in this special case, 
5 ip) — 2.18 
qd | i ( ) 


PES yer 


We prove this statement by induction on n. For n = 1, the statement is 
trivially true as [ol = Fal = 1. Now let us assume the statement is true for 
n—1, and prove it for n. A multiset permutation of K’ either ends in a 2, and 
then its last entry is not involved in any inversion, or it ends in a 1, and then 
its last entry is involved in exactly ag = n — ay inversions. By the induction 


hypothesis, this means that 


S- gi?) = Ps | Ee ame bee 
a a,—1 


pES ger 1 


By (2.16), it is now easy to see that the right-hand side is in fact equivalent 
to pe , completing the induction proof of (2.18). 

We are now in a position to prove our theorem in its general form. We will 
do this by induction on k, the case of k = 1 being trivial, and the case of 
k; = 2 being solved above. Let us assume that the statement of the theorem 
is true for K = {1%,2%,--- ,k%}, and prove that then it is also true for 
Kt = {1% , 29)... ko, (k+ 1)4e41}, 

Note that any permutation of K* is completely determined by the pair 
(p',p”), where p’ is the multiset-permutation obtained from p by replacing all 
entries less than k + 1 by 1, and p” is the permutation obtained from p by 
removing all copies of k +1. It is then clear that 


i(p) = ip") + i(p"), 
and that p’ and p” are independent of each other. 
Then the problem of finding Diy q') is clearly equivalent to the previous 
special case, and therefore we get that al gi’) = Ar 


Now let us find ae qi(?"). If we remove all copies of k + 1, we can apply 
the induction hypothesis, and see that 


pl! - ai a2 ak : 


Finally, as any p’ (consisting of a, + a2 +---+ a,x copies of 1, and ag41 
copies of k + 1) can be paired with any p” (consisting of a; copies of i for 
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i € [k]), it follows that 


S- gil?) = S- gil?) ; y gi?”) 
p’ pl 


PES gr 
eee Ure nea 
Aak4+1 ai ag ak 


n 


Ps 


‘eal 
Here the last equation is immediate from (2.14). This completes our induction 
proof. | 


2.2.3. Major Index and Permutations of Multisets 


Recall that for permutations of the set [n], we found in Theorem 2.18 that 
the statistics 1 and maj were equidistributed. We would like to see whether 
something similar is true for permutations of multisets. In order to be able 
to do that, we need to define the major index of multiset permutations. As a 
first step to that end, we need to define descents of multiset permutations. 

Fortunately, both of these definitions are what one expects them to be. If 
Pp = Pip2:*:Pn is a permutation of a multiset, then we say that 7 is a descent 
of p if pj > pj4i1. Similarly, the major index of the multiset permutation p is 
defined by maj(p) = Vien? 

Now we are ready to state the q-generalization of Theorem 2.18. 


THEOREM 2.30 
Let K = {1% ,2%,.-- ,k%} be a multiset so that ay +ag+---+a, =n. Then 
the statistics 1 and maj are equidistributed on the set Sx of all permutations 
of K. In other words, 


- gil?) = s- gmail) a ee asl 


peEesrK peEsK 


PROOF ~ Theorem 2.29 shows that the far-left side and the far-right side 
are equal. Therefore, it suffices to show that the sum in the middle is also 
equal to the far-left side. That, in turn, can be proved injectively, in a very 
similar manner Theorem 2.18 that covers the case in which a; = 1 for all i. 

The maps f and ¢, and the permutation w, can be defined just as in the 
proof of Theorem 2.18. The only differences are the following. 


1. An entry p; is large if it pj > py, and an entry p; is small if i A n, but 


2. When p,_, is a small entry, then the application of ¢ causes each large 
entry of p to be passed by small entry. So, the decrease of the number 
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of inversions caused by these movements is equal to the number of large 
entries, that is, aan aj; =n—)~)_, aj. On the other hand, affixing p, 
to the end of f(wp) creates the same number of non-inversions, proving 
that 


holds again. 


3. When p,,_; is a large entry, then the application of ¢ causes each small 
entry of p to be passed by one large entry. So, the increase of the 
number of inversions caused by these movements is equal to the number 


of small entries, that is, oom ai) — 1, where p, = j. On the other 


hand, affixing p, to the end of f(w,) creates sae a; =n—- ok Qi 
additional inversions. Therefore, 


just as in (2.9). 
i 


As we have mentioned, there are many interesting occurrences of Gaussian 
coefficients in combinatorics. Perhaps the most direct one is the following. 


THEOREM 2.31 
Let q be a power of a prime number, and let V be an n-dimensional vector 
space over the q-element field. Then the number of k-dimensional subspaces 


of V is [P]. 


PROOF First, let us choose a k-tuple of vectors in V that form an 
(ordered) basis for a k-dimensional subspace. For this, we have to choose 
k linearly independent vectors from our vector space V. For the first basis 
vector vj, we can choose any vector in V except the zero vector, so we have 
q” — 1 choices. For the second basis vector, we cannot choose any multiples 
of v;, therefore we have only q” — q choices. For the third vector, we cannot 
choose any of the q? possible linear combinations of v; and v2, yielding g” — q? 
choices, and so on. Iterating this argument, we see that we have 


(q” —1)(q" — q)---(q" — 4°") (2.19) 


choices for an ordered basis of a k-dimensional subspace of V. It goes without 
saying that any such subspace has many ordered bases. In fact, repeating the 
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above argument with k playing the role of n shows that the number of ordered 
bases of a k-dimensional subspace is 


(q* — 1)(q* — q)---(q* — g*7?). 


So this is how many times each k-dimensional subset of V is counted by (2.19). 
Therefore, the number of such subspaces is 
ig = Dig =e =a*)_ (t= 1g? 1). (gt = 1) 
(a? 1) g* gy eek gh gh") (@® L(g? — 1) eee(g— 1) 
— [nl[n—1)---[(n—k41] © ] 
[k]! k} 


We will see some alternative interpretations of the Gaussian coefficients in 
the exercises. 


Exercises 
1. (+) Let us generalize the notion of Eulerian polynomials as follows. Let 
B=} (ara, 
pESn 


That is, the only difference between this definition and that of A,(z) is 
that here the parity of the number of inversions is taken into account. 
Prove that 

Bon(z) = (1 — 2)” An(z), 


and 
Bon +i(Z) = (1 — z)”An+1(2)- 


2. Following the line of thinking found in Exercise 17 of Chapter 1, define 
the r-major index of p, denoted by rmaj(p) as 


rmaj(p)={ S> i) +{G7): 1Si< jn, pi>pjs>pi-r}l 
i€RD(p) 


where RD(p) denotes the set of all r-falls of p, as defined in the men- 
tioned exercise. 


(a) Explain why the r-major index is a generalization of both the num- 
ber of inversions and the major index. 
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10. 


11. 
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(b) Prove that for any positive integers r, the r-major index is a ma- 
honian statistic. 


Prove (without using the general formula for b(n,k)) that b(n,3) = 


(H)— (0) itn>3. 


Prove (without using the general formula for b(n,k)) that b(n,4) = 
("7") — ("3") ifn > 4. 

(-) Let p = pip2--+Pn, and let us call (p;,p;) a 2-inversion if i < J, 
and pj > pj + 1. What is the total number of all 2-inversions in all 
permutations of length n? 


Let m > 1 be a positive integer, and let 7 be a nonnegative integer, 
with 7 < m. Prove that if n is large enough, then the number of n- 
permutations p for which i(p) = 7 mod(m) is independent of J. 


Let T be a rooted tree with root 0 and non-root vertex set [n]. Define 
an inversion of T to be a pair (7,7) of vertices so that 1 > j, and the 
unique path from 0 to 7 goes through 7. How many such trees have zero 
inversions? 


Let p € S, have n — 2 descents. What is the minimal and maximal 
possible value of i(p)? 


It follows from Lemma 2.12 that 


S~> p(n—a(j)) = Sy p(n —a(j)), 


j even j odd 
where a(j) = (37? + j)/2. Find a direct bijective proof of this identity. 


Let p = pip2--:Pn be a permutation, and let our goal be to eliminate 
all four-tuples of entries (pa, Pb, Pc; Pa) in which a < b < c < d and 
Pa < Pe < Po < pa. In order to achieve that goal, we use the following 
algorithm. We choose a four-tuple F' with the above property at random, 
and interchange its two middle entries. By doing that, we took away 
the undesirable property of F', but we may have created new four-tuples 
with that property. Then pick another four-tuple with that property, 
and repeat the procedure. 


Prove that no matter what p is, and how we choose our four-tuples, this 
algorithm will always stop, that is, it will eliminate all four-tuples with 
the undesirable property. 


Is it true that b(n, k) is a polynomially recursive function of n for any 
fixed k? (Polynomially recursive functions are defined in Exercise 29 of 
Chapter 1.) 
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12. 


13. 


14. 


15. 


16. 
17. 


18. 


19. 


20. 


21. 


Let p(n, k) be the number of partitions of n into k parts. Let P(z) = 
yu, p(n, k)z*. Does there exist an integer n > 2 so that P(z) has real 
zeros only? 


Let B,, be the set of all n-tuples (61, bo,--+ ,b,») of non-negative integers 
that satisfy b; < 7—1 for all 7. How many elements of B, satisfy 


Th bi = BP 


Let B,, be defined as in Exercise 13, and let B(n,k) be the number of 
n-tuples in B, that have exactly k different entries. Find a formula for 
B(n,k). 


Express b(n, &) using summands of the type b(n — 1,72). 


Compute the value of sl) (—ayko(n, k). 


Let p € S;, have n—1 alternating runs, and let us assume that n = 2k+1. 
What is the minimal and maximal possible value of i(p)? 


(a) (+) Let A = {1%,2%}, and let us assume that a, and ag are 
relative primes to each other, with a; + a2 =n. Let I(A,k) be the 
number of permutations p of A so that 


i(p) =k mod n. 
Prove that I(A,k) = +(,") for all k. 


ay 
(b) (+) What can we say about I(A,k) if a, and az have largest com- 
mon divisor d > 1? 


(+) The Denert statistic, denoted by den, is defined on S;, as follows. 
Let p € S;,,, then den(p) is the number of pairs (¢, k) of integers satisfying 
1<€<k<n, and one of the conditions listed below 

Dk < pe Sk, 

pe<k <p. 

k < pr < pe. 


So for instance, den(132) = 2 as the pair (2,3) satisfies the first condi- 
tion, and the pair (1,2) satisfies the second condition. Prove that the 
Denert statistic is Mahonian. 


We know that (7) is the number of northeastern lattice paths from 
(0,0) to (k,n —k). Extend this correspondence to one that provides an 
interpretation for cae 


Prove by way of computation that 


ele palma 
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23. 


24. 


25. 


26. 
27. 


28. 


29. 


30. 
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Prove the identity of the previous exercise by a combinatorial argument. 


Prove that 


| =qr'. | a roa [tote hee |+[no al 


n 


| is always a polynomial with 


Prove by way of computation that [ 
non-negative integers as coefficients. 


Prove that 


Py] = Deen), 


n>0 


where p(i, k,n) is the number of partitions of the integer n into at most 
zi parts of size at most k each. 


For what values of n and k will [2] have log-concave coefficients? 


Let m <n, and let 
Am(n) = {1,1,2,2,--- sm,m,mt+1,m+2,--: ie 
Let @m(n) be the number of all permutations of the multiset A,,(n) in 
which 12---n occurs as a subword. (The letters of this subword do not 
have to be consecutive entries of the permutation.) Prove that 
Am4i(n) = (n+ 2m)am(n) — m(n + m)am—1(n). 

Letn<k< Gy Prove that 

b(n+ 1,k) = b(n+1,k — 1) + b(n, k) — b(n, k —n—- 1). 


(+) Find a formula for 


sa [§] 


k=0 


(+) Consider the following refinement of the Eulerian polynomials. Let 


An,e(q) = Sg"), 
p 


where the sum is taken over all n-permutations having k — 1 descents. 
These polynomials are often called the g-Eulerian polynomials. Prove 


that 2 
f= > Analo|* 2"): 
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3l. 


32. 


33. 


34. 


35. 


Let p be a permutation of length n — 1. Insert the entry n into all p 
in all possible ways. This yields n distinct permutations of length n. 
Compute the major index of each of these permutations. Prove that all 
these n major indices will be distinct, and that their set will be the set 
of integers in the interval [maj(p), maj(p) +n — 1]. 


Use the result of the previous exercise to give an induction proof of 
Theorem 2.18, that is, of the fact that 7 and maj are equidistributed. 


A simple graph G on vertex set [n] is called the inversion graph of the 
n-permutation p if ij is an edge of G if and only if (7,7) is an inversion 
of G. Find an example for an unlabeled graph U that is not an inversion 
graph of any permutation. Try to find an example with as few vertices 
as possible. 


Let p = pip2:-:pn be an n-permutation, and let G(p) be the inversion 
graph of p as defined in the previous exercise. Let 1 < 7 < k be three 
elements of [n], and interchange the strings p;---pj;—1 and p;---pr—1 of 
p, to get the permutation 


p = P1-** Pi-1Pj °° * Pk—-1Pi*** Pj-1Pk*** Pn- 
Describe G(p’) in terms of G(p). 


A graph G on vertex set {@1,@2,--- ,@n} is called a comparability graph 
if there exists a poset P on vertex set {a1,a2,--- ,a,} so that (a;,a;) 
is an edge in G if and only if a; and a; are comparable elements in P. 
Prove that all inversion graphs are comparability graphs, but not all 
finite comparability graphs are inversion graphs. 


Problems Plus 


il, 


Prove that the polynomials [,(z) have log-concave coefficients without 
using generating functions. 


. Let 


Exp,(2) = (2) = 


! 
SoC! 
Note that for g = 1, the power series Exp,(z) reduces to the power series 


gee ae 
indo ar = e. 


Express the bivariate generating function 
1+ YH atin 
n>1 pESn 


in terms of Exp,. 
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gn 


. Let exp,(z) = Vin>o Ta: «Express the bivariate generating function 


Ea 3e3 ae, 


n>1pESn 


in terms of exp,. 


. Let a(k, 2) be the number of n-permutations having & descents and ma- 


jor index @. Let d(k,2) be the number of n-permutations gq having k 
excedances and satisfying den(q) = @. Prove that a(k, €) = d(k, ). This 
fact can be referred to by saying that the (den, exc) statistic is Euler- 
Mahonian. 


. Find a permutation statistic s : 5S, + N so that the number c(k, ¢) of 


n-permutations p for which s(p) = k and i(p) = @ is equal to the number 
a(k,€) defined in Problem Plus . In other words, find a statistic s so 
that the joint statistic (s,7) is Euler-Mahonian. 


. Prove that the joint statistic (dmc, maj) is Euler-Mahonian. See Exer- 


cise 10 of Chapter 1 for the definition of the statistic dmc. 


. Let 1<k <n. Prove, using the polynomial [,,(z), that the number of 


n-permutations p for which 
maj(p)=j( mod k) 


does not depend on 7. 


. Define the (q,7)-Eulerian polynomials by 


Af, kyr] = So g"™ase), 


PESn, 
where rmaj(p) is the r-major index of p. Prove that 


Aln, k,r] = [r + k]A[n—1,k,r] +q**"-1 [m4 1—k—r]A[n—1,k-1,7]. 


. A parking function is a function f : [n] > [n] so that for all i € [n], there 


are at least i elements 7 € [n] for which f(j) <7. Prove that the number 
of parking functions on [n] satisfying et t(j) = (5) —& is equal to 
the number of rooted trees with root 0 and non-root vertex set [n] that 
have k inversions. (See Exercise 7 for a definition of an inversion in a 
tree.) 


Prove that the Gaussian polynomial [7] has unimodal coefficients. 
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11. 


12. 


13. 


14. 


15. 


Let A = {1%,2°?}, and let d be the largest common divisor of a; and 
ag. Now let J(A,k) be the number of permutations p of A that have 
first entry 1, and for which 


i(p) =k mod ay 


holds. Let 0<t < 4 — 1. Prove that 


H(A,b) == ("7 1), 


ay, a,—1 


Note the difference from Exercise 18. Here we are looking at residue 
classes modulo a;, not modulo n. 


Log-concavity is a concept for sequences of numbers, but it can be ex- 
tended to a concept for sequences of polynomials as follows. 


Let po(q), pi(@),°-: ;Pm(q) be a sequence of polynomials with nonnega- 
tive coefficients. We say that this sequence is q-log-concave if the polyno- 
mial p7(q)— pr—1(¢)pr+i1(q) has non-negative coefficients for all k. Prove 
that for any fixed n, the sequence of polynomials als [7] pitty bal is 
q-log-concave. 


Let us extend the notion of unimodality to polynomials as follows. Let 
po(q),P1(q),;°** ;Pm(q) be a sequence of polynomials with non-negative 
coefficients. We say that this sequence is q-unimodal if there exists an 
index j so that 0 < 7 < m and for all 2, the polynomial p;(q) — pi(q) 
has non-negative coefficients. Note that a g-log-concave sequence does 
not have to be q-unimodal. Prove that for any fixed n, the sequence of 
polynomials [$], [7],--- [2] is q-unimodal. 


Generalize the result of Exercise 27 to the multiset 
Amynyr = fie, a, seas Ra (m az 1)"; ee wn" 


as follows. Let am,,(n) be the number of permutations of Amn,- that 
contain a subword consisting of r copies of 1, then r copies of 2, and so 
on, ending with r copies of n. Again, the letters of the subword do not 
have to be consecutive entries in the permutation. Prove that 

Om+i,r(n) = (rn + 2m — 7 + Lam y(n) — m(rn + M)am—1,r(n), 
where ao,-(n) = 1, and a,,-(n) =rn + 1—r. 


Let us call a 2n-permutation p = p,p2--- pon 2-ordered if py < p3 < 
+++ < Pon—1 and po < pa < +++ < pan. Prove that 


So i(p) = nar, 


where the sum is taken over all 2-ordered 2n-permutations p. 
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Solutions to Problems Plus 
1. This result is proved by a recursively built injection in [58]. 


2. Richard Stanley [290] proved that 


F 1-t 
if Fe |) ia a er 
ss In]! Exp,(u(t—1) — 8) 


n>1pESn 


Note that in this problem and the next, the exponent of t is the num- 
ber of descents (respectively, excedances), and not that parameter plus 
one. Therefore, setting g = 1, we do not get the exponential generat- 
ing function of the Eulerian polynomials A,(z), but of the polynomials 
An(z)/z. 


3. John Shareshian and Michelle Wachs [283] proved that 


Ly Se gexe(p) gimai(p) WU _ _ Um tgexp,tz) 


See. [n|!  exp,(ztq) — tq exp, (z) : 


We point out that generating functions in which z” is divided by [n]! 
are sometimes called g-exponential generating functions. 


4. This result is due to Dominique Foata and Doron Zeilberger [174], who 
proved it by providing alternative interpretations for the Denert statis- 
tic. In particular, they showed that 


den(p) = 1, +ig+++++im +i(EBacp) + i(Nezcp), 


where 71, 72,°-: , tm are the excedances of p, while Excp is the substring 
Di, Pin’ **Di,,, and Nexcp is the substring obtained from p by removing 
Exc p. For our example in the text, the permutation 132, we get that 


den(132) =2+0+0=2, 
as we should. 
5. Such a statistic was given by Mark Skandera in [285]. 


6. This result was obtained by Dominique Foata and Guo-Niu Han [175], 
who first proved that the joint statistics (d,i) and (d, maj) had the same 
distribution. 


7. For k =n, the statement means that there are (n — 1)! permutations in 
Si, so that maj(p) = j7( mod n). This result was first proved in [29], 
using a heavy algebraic machinery. In that same paper, the authors 
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10. 


11. 


provided a bijective proof as well, but that still used Standard Young 
Tableaux and the Robinson—Schensted correspondence, which we will 
cover in Chapter 7. The general statement for k € [n] was given the 
following simple and beautiful proof in [30]. We know from Theorems 
2.3 and 2.18 that 


So 2s?) = I, (2) = (1+ 2) +2427)--(Lt2t---+2"1). 
pESn 


If we count our permutations according to the remainder of the major 
index modulo k, then we have to consider the above equation modulo 
the polynomial z*—1. If z is any kth root of unity other than 1, then the 
left-hand side vanishes as there is at least one factor (I+-z+-+-+2*7!) = 
aol on the right-hand side, which vanishes. Therefore, 
In(z) = $0 2s) = o(14+24---+2*1) mod (z* —1) (2.20) 
PESn, 


as the left-hand side and the right-hand side have k — 1 common roots, 
namely those k — 1 roots of unity. Setting z = 1, we get that c = n!/k. 
Therefore (2.20) implies that 


! 
In(z) = + (Lt 2+e-2®) mod (z* — 1). 


That proves that the number of n-permutations p satisfying 
maj(p) =j (| mod k) 


is n!/k, proving our claim. 


. This result is due to Don Rawlings [27]]. 


. This result, in a slightly different form, was found by Germain Kreweras 


[236]. 


There are several proofs of this fact that had first been noticed by Cayley 
at the end of the nineteenth century. Some of these proofs are reasonably 
short, but use sophisticated machinery. See [279], or [269] for such 
proofs. An elementary proof was given by Kathy O’Hara [258], who 
used the subset sum interpretation of Gaussian coefficients in her proof. 
Her argument was later explained in an expository article by Zeilberger 
[336]. 


This result was proved in [102]. The authors showed that if p = pip2--- Pn, 
then exactly d of the n cyclic translates pjpo---DPn, P2°**PnP1, ***; 
PnP1***Pn—1 have first entry 1 and inversion number & modulo a,. As 
£(274) = 4(7), this proves the result. Note that the result is identi- 
cal to the result of Exercise 18 (b), even if in that exercise we counted 
different permutations. 
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This result was proved in [105]. 


This is a special case of a more general result of Lynne Butler [106], 
which is of group-theoretical flavor. In her proof, Butler uses the in- 
teresting fact that the number of subgroups of order q” of the Abelian 
group Z” is the Gaussian polynomial |], where q is a prime. 


It is proved in [191] that the exponential generating function of the 
SEQUENCE Gm,r(7) is 


(12) exp ( ae ) | 


1l-z 


from which the proof of our statement follows. In [333], Lilly Yen 
sketches a bijective proof. 


A bijective proof of this result is due to Volker Strehl, and can be found 
in [306]. 


3 


In Many Circles. Permutations as Products 
of Cycles. 


3.1 Decomposing a Permutation into Cycles 


So far we have looked at permutations as linear orders, that is, ways of listing 
n objects so that each object gets listed exactly once. In this section we will 
discuss permutations from a different viewpoint. We will consider them as 
functions. Let us redefine permutations along these lines. 


DEFINITION 3.1 Let f : [n] > [n] be a bijection. Then we say that f 
is a permutation of the set [n]. 


This definition certainly does not contradict our former definition of permu- 
tations. Formerly, we said that 34152 was a permutation of length five. Now 
we can reformulate that sentence by saying that the function f : [5] > [5] 
defined by f(1) = 3, f(2) = 4, f(3) =1, f(4) = 5, and f(5) = 2 is a permu- 
tation of [5]. Going backwards, the one-line notation simply involved writing 
fC)f(2)--+ f(m) in one line. 

This new look at permutations makes another way of writing them plausible. 


We write 
12345 


P= 34152° 


expressing that f maps 1 to 3, 2 to 4, 3 to 1, 4 to 5, and 5 to 2. 

This notation is called the two-line notation of permutations. It is more 
cumbersome than the one-line notation, which consists of writing the second 
line only, but it has its own advantages as the reader will see shortly. 

Let f and g be two permutations of [n]. Then we can define their product 
f-g by (g- f)(@) = g(f(2)) for z € [n]. It is straightforward to verify that the set 
of all permutations of [n] forms a group when endowed with this operation. 
Therefore, the set of all permutations of [n] is often denoted by S,, and is 
called the symmetric group of degree n. We note that for n > 3, the group 
S,, is not commutative, so in general, fg 4 gf. 

The symmetric group is a quintessential ingredient of group theory. It is 
well known, for instance, that every finite group of n elements is a subgroup of 
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FIGURE 3.1 
The cycles of f = 34152. 


S,,. Extensive research of the symmetric group is therefore certainly justified. 
In this chapter, we will concentrate on the enumerative combinatorics of the 
symmetric group, that is, we are going to count permutations according to 
statistics that are relevant to this second way of looking at them. 

A closer look at our running example, the permutation f = 34152, reveals 
that f permutes the elements 3 and 1 among themselves, and the elements 
2, 4, 5 among themselves. That is, no matter how many times we apply f, 
we will always have f(3) = 1 or f(3) = 3, and f™(3) will never have any 
other values. In other words, f cyclically permutes 1 and 3, and f cyclically 
permutes 2, 4 and 5. This phenomenon is illustrated in Figure 3.1. 

The facts that we have just described are highlighted by a third way of 
writing the permutation f. We write f = (13)(245), and call it a cycle notation 
of f. When reading a permutation in this notation, we map each element 
to the one on its right, except for the elements that are last within their 
parentheses. Those are mapped to the first element within their parentheses. 


Example 3.2 
The permutation g = (12)(356)(4) is the permutation 
123456 
o— 915463’ 


O 


Several comments are in order. First, if the cycles are disjoint, the order 
among the cycles clearly does not matter, that is, (12)(356)(4) = (356) (4)(12) = 
(4)(12)(356), and so on. Indeed, the image of each i € [n] only depends on its 
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position within its cycle, and some other elements in its cycle. Second, if a 
cycle has at least three elements, then the order of elements within that cycle 
matters, up to a certain point. Indeed, the cycles (356), (563), and (635) 
describe the same action of g on elements 3, 5, and 6, but the cycle (365) 
describes a different action. This action maps 3 to 6, not to 5 as the previous 
actions did. A little further consideration shows that each cyclical action on k 
elements can be described by writing k different cycles as one can start with 
any of the k elements. 

We would like to have a unique way of writing our permutations using the 
cycle notation. Therefore, we will write the largest element of each cycle first, 
then we will arrange the cycles in increasing order of their first elements. This 
way of writing permutations will be called their canonical cycle notation. 


Example 3.3 
The permutation (312)(45)(8)(976) is in canonical cycle notation. 


3.1.1 Application: Sign and Determinants 


The cycle decomposition of a permutation f contains some crucial information 
about f. For instance, if we know the cycle lengths of f, we can compute the 
smallest positive integer m for which f™ is the identity permutation. This 
number m is called the order of f in group theory. Indeed, m is obtained as 
the smallest common multiples of all cycle lengths of f. 

A common way of fitting permutations into algebraic frameworks is by 
defining permutation matrices, that is, square matrices whose entries are all 
equal to 0 or 1, and that contain exactly one 1 in each row and each column. 
There are two equally useful ways to do this. Let p = pipo-:-pn be an 
n-permutation, and let Ap be the matrix n x n matrix in which 


fees lif R= I 
Ap(j, 4) = ‘i otherwise. 


It is then straightforward to check that the map f : S, — R”*” defined by 
f(p) = Ap is a homomorphism, that is, Apg = Ap- Ag. 


Example 3.4 
If p = 2413 = (4312), then we have 
0010 
1000 
oe 0001 
0100 
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Now let B, be the n x n matrix defined by 
Bp(i, 5) = . otherwise. 


The following example illustrates how closely the matrices A, and By, are 
connected. 


Example 3.5 
If p = 2413 = (4312), then we have 
0100 
0001 
Bvo=1 1000 
0010 


U 


Clearly, for any p, the matrix B, is the transpose of Ap. Furthermore, it 
is straightforward to check that A,B, = B,pAp = I, so the matrices Ap and 
By are inverses of each other. In other words, the inverse of a permutation 
matrix is its own transpose. 

Comparing the definitions of Ap and By, one could ask when is the first one 
easier to use, and when is the second one easier to use. The advantage of the 
first definition is that as we said, the map f : S, > R”*” defined by f(p) = 
A, is a homomorphism. This is not true for the map g: S, > R”*” defined 
by g(p) = Bp. That map is an anti-homomorphism, that is, Byg = By: Bp. 
This can be useful when we use our matrices to permute vectors of size n. 

For instance, let 

Beal 

x2 

v3 

v4 
If the permutation p = 2413 = (4312) acts on the coordinates of this vector, 
v2 
v4 
Ty 
v3 
by simply taking the product B,x. If now another permutation q acts on the 
vector p(x), then we can obtain the image by computing By(Bpx) = Bpq(x). 
If, instead of the column-vector x, we had worked with the row-vector y = 
(y1, y2, y3, ya), then we would have used the matrices A, and A,, to compute 
the images yA, and yApg = yApAg. 


it takes the vector x into p(x) = . We could have obtained this vector 


DEFINITION 3.6 A permutation is called odd (resp. even) if it has an 


odd (resp. even) number of inversions. 
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The following proposition shows that we have to be careful throwing around 
the words “odd” and “even” when describing permutations. Let us call a cycle 
even (resp. odd) if it consists of an even (resp. odd) number of elements. 


PROPOSITION 3.7 

Let p be a permutation that consists of one cycle, and let that cycle be of even 
length. Then p is an odd permutation. If p is a permutation that consist of 
one cycle, and that cycle is of odd length, then p is an even permutation. 


PROOF We prove these claims simultaneously, by induction on the length 

n of the only cycle of our permutation p. For n = 1 and n = 2, the statement 
is trivially true. Now let n > 3, and consider the cycle (pipo---pn). It is 
straightforward to verify that (pip2---Pn) = (pip2-+:Pn—1)(Pn—-1Pn). The 
multiplication by (pn—1pn) at the end simply swaps the last two entries of 
(p1p2-++Pn—1), and therefore, either increases the number of inversions by one, 
or decreases it by one. So in either case, it changes the parity of the number 
of inversions. The proof is then immediate by the induction hypothesis. 


The omnipresence of this notion is illustrated by the following Lemma. 


LEMMA _ 3.8 
Let p be a permutation. Then the following are equivalent. 


(i) p is even, 
(ii) det Ap = det B, = 1, 


(itt) the number of even cycles of p is even. 


PROOF 


e (i) = (it). This follows from the definition of determinants, as given 
in Theorem 2.21 because the determinant of a permutation matrix B, 
has only one nonzero term, and that is (—1)" []}_, big). The claim 
follows as each factor after the product sign is equal to 1. 


e (it) = (itt). This is true as p is the product of its cycles. By Proposition 
3.7, even cycles correspond to odd permutations, so the determinant of 
their permutation matrices is —1. Therefore, there has to be an even 
number of them for the determinant of their product to be 1. 


In particular, the product of two even permutations is even, and the inverse 
of an even permutation is even. As the identity permutation is even, this 
proves the following. 
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PROPOSITION 3.9 
The set of all even permutations in S,, forms a subgroup. 


This subgroup is called the alternating group of degree n, and is denoted 
by Alt,. The reader should prove at this point that Alt, has n!/2 elements 
if m > 2, then she should check her answer in Exercise 1. Like the symmetric 
group, the alternating group has been vigorously investigated throughout the 
last century. For instance, it is known that Alt, is a simple group if n > 5. 
In fact, among all finite simple groups, Alt, is the easiest to define, except 
for the cyclic groups Z,, where p is a prime. (See any introductory book on 
group theory for the definition of a simple group or Z,.) It is also interesting 
that Alt, is by far larger than any other proper subgroup of S,. Indeed, 
other proper subgroups of S,, are of size at most (n — 1)!. Can you find such 
a subgroup? 


3.1.2. An Application: Geometric Transformations 


Some crucial properties of geometric transformations are easy to read off if we 
decompose the corresponding permutation into the product of simpler permu- 
tations. To start, consider a regular hexagon H. Apply various symmetries 
to this hexagon. It is a well-known fact in group theory, and is not difficult to 
prove, that there are 12 such symmetries, and that they can all be obtained 
by repeated applications of r, which is a reflection through a fixed axis a, and 
t, which is a rotation by 60 degrees counterclockwise. See Figure 3.2 for an 
illustration. 

We point out that both transformations r and t correspond to permutations 
of the set {A,B,C,D,E,F}, or, after relabeling, [6]. In particular, r = 
(43)(52)(61), and ¢ = (123456). 

It is natural to ask the following question. Given a series of symmetries, such 
as s = trrtrtrrtr, how can we decide whether the composite transformation 
can be realized just by moving our hexagon in its original 2-dimensional plane? 

This question is easy to answer if we have the transformation given in the 
above form. Clearly, transformation r changes the orientation of the hexagon, 
and therefore cannot be realized by a 2-dimensional movement. On the other 
hand, t does not change the orientation of our hexagon. Consequently, if and 
only if our composite transformation contains an odd number of reflections r, 
then it changes the orientation of H, and therefore cannot be realized by a 
2-dimensional movement. The example s of the previous paragraph contains 
six reflections, and is therefore realizable in the plane. 

Similar considerations are helpful in deciding whether a certain symmetry 
of a 3-dimensional solid, such as a cube, can be realized by moving the cube 
in the 3-dimensional space. 
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FIGURE 3.2 
Two symmetries of a regular hexagon. 


3.2 Type and Stirling Numbers 


We start with two well-known and basic enumeration problems. It is natural 
to ask how many n-permutations have a given cycle structure — for instance, 
how many 12-permutations consist of a 4-cycle, two 3-cycles, one 2-cycle, and 
zero 1-cycles. It is also natural to ask the more inclusive question of how 
many n-permutations have exactly k cycles. 

In order to facilitate the answer of the first question, we make the following 
definition. 


3.2.1 Cycle Type of a Permutation 


DEFINITION 3.10 Let p be an n-permutation that has exactly a; cycles 
of length i, for all positive integers i € [n]. Then we say that p is of cycle 
type (a1, @2,+++ ,@n). 
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Example 3.11 
The permutation p = (21)(534)(6)(987) is of cycle type (1, 1,2,0,0,0,0, 0,0). 


The number of permutations with a given cycle type is fairly easy to obtain. 


PROPOSITION 3.12 


Let (a1, d2,°-* ,Qn) be an n-tuple of nonnegative integers satisfying the equa- 
tion Sy, a;-i = n. Then the number of n-permutations of cycle type 
(a1, a2, a , Gn) is 

n! 


PROOF Let us write the elements of [n] in a linear order, in one of n! 
possible ways. Then let us place parentheses between the numbers so that the 
first a, entries form the a; cycles of length one, the next az entries form the 
ag cycles of length two, and so on. The permutations we obtain this way will 
all be of cycle type (a1, @2,--- ,@,), but they will not all be different. Indeed, 
the sets of entries forming cycles of the same length can be permuted among 
each other without changing the resulting permutation. Therefore, we obtain 
each permutation from a;!a!---a,! linear orders, due to permuting cycles of 
the same length. Finally, each 7-cycle can be obtained in 7 different ways as 
any of its 7 entries can be at the first position. So, even if we keep the sets of 
entries in each cycle fixed, there are 1%12°? ---n°” different linear orders that 
could lead to any given permutation of cycle type (a1, d2,--- ,@n). This shows 
that on the whole, each permutation of the desired cycle type is obtained from 
ay!ag!+++ay,!1%2% ---n linear orders, and the statement is proved. 


For instance, there are (n— 1)! permutations of length n that consist of one 
(2n)! 
nian 


n-cycle, and there are permutations of length 2n that consist of n cycles 


of length two. 


3.2.2. Application: Conjugate Permutations 


In the symmetric group S,, two permutations g and h are called conjugates 
of each other if there exists an element f € S, so that fgf~! = h holds. In 
group theory, it is often very useful to know that two elements are conjugates 
as they share many basic properties. Fortunately, the cycle decomposition of 
g and h reveals whether they are conjugates or not. This is the content of the 
next lemma. 


LEMMA _ 3.13 
Elements g and h of S, are conjugates in S,, if and only if they are of the 
same cycle type. 
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PROOF Recall that if g and h are two n-permutations, then the action 
of gh on [n] is obtained by first applying g to [n], and then by applying h to 
the output. 

First let us assume that g and h are conjugates, that is, fgf—! = h for 
some f € Sn. Let (bib2---bx) be a cycle of g. Then g’(b1) = bj41 holds 
for all indices i € [k — 1], and g*(b1) = bi. Ash = fgf—, we have h' = 
fof! fof? faf-! = fgif-}. Therefore, hi(x) = (fg f-)(a). 

Now choose x so that f(x) = bi. Then (fg'f~+)(x) = f7'(g‘(bi)) = 
f+ (bi41) if i € [k — 1], and (fg*f-!)(z) = f-1(b1). Thus multiplying by f 
from the left and f~! by the right turns the cycle (b1b2--+b,) into the cycle 
(f~1(b1) f7*(b2) +--+ f7'(be)). Therefore, the k-cycles of g are in bijection with 
the k-cycles of h for all k, and the “only if” part of our lemma is proved. 

Now let us assume that g and h have the same cycle type. We construct a 
permutation f so that fgf—! = h. If (bib2-++ by) is a cycle of g and (cc: -- cx) 
is a cycle of h, then the argument of the previous paragraph shows that we 
must choose f so that f~'(b;) = c; for i € [k]. This defines f~! for k entries. 
To find f—! for the remaining n — k entries, proceed similarly for all the 
remaining cycles. 


Remark We point out that in the second paragraph of this proof we 
showed that conjugating by f turned the cycle (b1b2---b,) into the cycle 
(f~1(b1)f7*(b2)++-f71(b,)). This simple fact is often used in similar argu- 
ments. We will see one application in the next subsection. 

It is straightforward to prove that the relation “g and h are conjugates” is 
an equivalence relation. The equivalence classes created by this relation are 
called conjugacy classes. The following simple consequence of the previous 
lemma is of fundamental importance in representation theory. 


COROLLARY 3.14 
The number of conjugacy classes of Sn, is p(n). 


3.2.3 Application: Trees and Transpositions 


We have seen that the decomposition of a permutation into disjoint cycles is 
unique, up to the transformations discussed above. We have also seen that the 
multiplication of disjoint cycles (viewed as permutations) is a commutative 
operation. However, if we drop the condition that the cycles be disjoint, 
everything falls apart. Indeed, we have (12)(13) = 312 and (12)(13) = 312 4 
(13)(12) = 231 as counterexamples. In fact, it is not hard to see that any 
permutation in S, can be obtained as a product of (not necessarily disjoint) 
2-cycles, or, in other words, transpositions. See Exercise 11 for a stronger 
version of this fact. 
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While the topic of generating permutations from given sets of not neces- 
sarily disjoint cycles belongs more to group theory than to combinatorics, the 
following results of Janos Dénes [140] certainly have a combinatorial flavor. 
Let us call a permutation cyclic if it consists of one cycle only. 


LEMMA 3.15 


Let 82, 83,°+* ,8n be transpositions of S,. Then the product sn5n—1°+++ 82 is 
equal to a cyclic permutation if and only if the graph G(s2,83,-+- ,8n) with 
vertices 1,2,--- ,n and edges 52, 83,-++ , Sn 1s a tree. 


In the unlikely event that the reader is not familiar with basic graph theory, 
the relevant definitions can be found in any introductory combinatorics book, 
such as [74]. 


PROOF Let us assume that s,5,-1--+:+s2 is cyclic. Clearly, for any 
i,m € [n], there is a path between i and 8,8 -1:-+S2(t) in our graph. As 
all entries j € [n] are in the cycle 5,8 1--+ $2, choosing the right m gives us 
a path between i and any j. This means that G(s2, 53,--- , $n) is connected. 
On the other hand, by definition, our graph has n — 1 edges; therefore it has 
to be a tree. 

Now let us assume that T is a tree with vertex set [n] and with edges 
$2, 83,°** , Sn. Remove the edge s,,, then T splits into two smaller trees, say 
T’ and T”. By induction, each of these trees corresponds to a cycle, say C; 
and C,. As these cycles are disjoint, we have 


SnSn—15n—-2°°° $2 = (a1 a2) “CL: C2, 


where 8, = (a1a@2), with a; € Cy and ag € Cy. These conditions imply that 
Sn 8n—1°*+ 82 is a cyclic permutation. i 


One of the best-known theorems of graph theory is Cayley’s formula, stating 
that the number of elements of the set Ty, of all trees on vertex set [n] is n"~?. 
See [74] for some proofs. The previous lemma enables us to link this result to 
cyclic permutations. 


THEOREM 3.16 
Let p be a given cyclic permutation of length n. Then the number of ways to 
decompose p into the product of n —1 transpositions is n™~?. 


This result was first proved by Dénes [140]. The proof we are presenting is 
due to Moszkowski [253]. 


PROOF Clearly, the choice of p is irrelevant as one can always relabel 
the entries of p, so we may assume that p = (123---n). We are going to 
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FIGURE 3.3 
Labeling the edges of T. 


construct a bijection h : T, — Cy, where Cy is the set of all (n — 1)-tuples 
(Sn; Sn—1,°** , 2) of transpositions satisfying $7) S,—15n—2+++ $2 = (12---n). 

Let T € Ty. Then for every vertex i € {2,3,---,n}, there is exactly one 
path from 1 toi in T. If (a,2) is the last edge in this path, then we label the 
edge (a,i) by s?. See Figure 3.3 for an example. 

Let s; denote the transposition interchanging the two endpoints of s;. Set 
Cr = Sn Spn_1" -++ 827; then Cr is a cyclic permutation because of the previ- 
ous lemma. As an intermediate step in constructing our bijection h, we define 
the permutation fr by the formula 


fr(k) = Cp *(1), 
for 1 <k<n. The reader is invited to verify that 
fr Op: fp! = (123-7). (3.1) 
An explanation is given in the solution of Exercise 15. Moreover, we set 
u; = frsi’ fr (3.2) 


for 2<i<n. By Lemma 3.13, we see that u? is also a transposition as it is 
the conjugate of transposition s;7. Furthermore, 


ue uty ud = fr-Sn’ + fp’ + frsn-1) + fp! s+: fr se" + fr! 
= frSn' Sp—1" +++ 82" fr! = (12-+-n), 
where the last line holds because of (3.1). So (uZ,uz_,,---,u3) € Cn. Now 
we define h(T) = (uz ,uZ_,,--» , ud). 


We claim that h: Ty — Cy is a bijection. To see this, we will show that 
h has an inverse; in other words, that every element of C, has exactly one 
preimage under h. 
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FIGURE 3.4 
The connection between the labels of the two trees. 


(A) First we show that C € Cy cannot have more than one preimage. Let us 


assume that (Un,Un—1,°°* ,W2) € Cy. Then un: Un—1-+- ug = (12---n), 
and by the previous lemma, the graph G(un, Un_1,°-* , U2) with vertex 
set [n] and edges un, Un—1,°+* , Ug is a tree. 


(i) 


If T is a tree so that h(T’) = un: Un_1 +--+ Ug, then the transpositions 
8; are uniquely determined by the u; and formula (3.2). In fact, it is 
straightforward to check that the trees T and G(un, Un—1,°*: , U2) 
are isomorphic as unlabeled trees because T is obtained from the 
tree G(Un, Un—1,‘** ,U2) by applying the permutation fr to the 
vertex set [n]. Indeed, we get 


si’ = frluifr, 


and as we pointed out in the Remark after the proof of Lemma 
3.13, the effect of conjugating a cyclic permutation (in this case, a 
transposition) by fp ' is precisely the same as applying fr to the 
underlying set. 

On the other hand, we have fr(1) by definition, and we also know 
that for i > 1, the last edge of the path from 1 to i in T is s?. 
Therefore, if in G(un, Un—1,°+* , U2), the endpoint of the path from 
1 to the edge u; is x;, then we must have fr(a;) = 7. 


See Figure 3.4 for an illustration of these labels. 


We have seen in part (i) that T as an unlabeled tree is uniquely deter- 
mined by (Un, Un—1,°** , U2), and then we have seen in part (ii) that the 
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labels of T are also uniquely determined by (un,Un—1,°*: ,U2). So ifa 
T satisfying h(T) = (un, Un—1,°+* , U2) exists, it must be unique. 


8 


Now we show that each (tn, Un—1,++* , U2) € Cy has a preimage under h. 
Let T(un,Un—1,°** Ug) be the tree obtained from G(un, Un—1,°++ , U2) 
by applying the permutation f defined by f(x;) = 7% to the vertices, for 
2<i<n. (We know from part (A) that this is the only tree that has 
a chance to be the preimage of (tn, tUn—1,°+: ,U2).) Here 2; is defined 
as in (ii) of part (A). We will show that h(T) = (un, Un—1,°-+ , U2), 80 
T is the preimage we are looking for. By construction, the edges of T 
are the f~'u;f. For brevity, let us write s; = f~!u;f. Then we can 
compute Cp for our candidate T as follows. 


Cr = SpSp—1'+ 82 = f-' + (12+--n)- f = (f()F(2)--- f(n)), 
aS Un * Un—1 +++ Ug = (12---n). As the next step, we compute fr(k) for 


fas 
fr(k) = Ce *(1) = f(k). 


Therefore, fr = f, so indeed, h(T) = (Un, Un—1,-++ , U2) as claimed. 


3.2.4 Permutations with a Given Number of Cycles 


Let us return to the enumeration of permutations with conditions on the 
number of their cycles. 


DEFINITION 3.17 The number of n-permutations with k cycles is 
called the signless Stirling number of the first kind, and is denoted by c(n,k). 


Example 3.18 
For all positive integers n > 1, the equality c(n,n — 2) = 2(%) + $(3) ("5") 
holds. 


SOLUTION _ There are two ways an n-permutation can have n—2 cycles. 
It can have a 3-cycle and n — 8 fixed points, or it can have two 2-cycles, and 
n —A fixed points. 


All we need to explain about the name “signless Stirling numbers” is the 
part “signless” and the part “Stirling numbers”. That is, what sign are we 
missing, and what is the connection between these numbers and the Stirling 
numbers of the second kind, defined in Chapter 1? We will answer these 
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questions shortly. Let us start with the most basic property of the signless 
Stirling numbers of the first kind. 


LEMMA 3.19 
Let us set c(n,0) = 0 ifn > 1, and c(0,0) = 1. Then the numbers c(n, k) 
satisfy 


c(n,k) = e(n—-1,k -—1)+ (n—1)c(n—-1,k). 


PROOF Take an n-permutation p with k cycles. Then the entry n of 
p either forms its own 1-cycle, and then there are c(n — 1,k — 1) possibilities 
for the rest of the permutation, or is part of a larger cycle, and then the 
permutation maps it into one of the other n — 1 elements. In this case, there 
are c(n — 1,k) possibilities for the permutation that we obtain if we omit n 
from p, and the result follows. 


The Stirling numbers c(n,0), c(n, 1),--- ,c(m,n) can be easily generated as 
the coefficients of a certain polynomial. This is the content of the next theo- 
rem. 


THEOREM 3.20 
For all positive integers n, the equality 
2(z+1)---(zg+n—-1)=9Y e(n,k)z* (3.3) 
k=0 


holds. In other words, c(n,k) is the coefficient of z® in z(z+1)---(z+n—1). 


PROOF Let b(n,k) be the coefficient of z* in the polynomial F,,(z) = 
z(z+1)---(¢+n-—1). It is then clear that b(n, 0) = 0 for n > 0, and we set 
b(0,0) = 1. We claim that the numbers b(n,k) satisfy the same recurrence 
relation as the numbers c(n, k), that is, 


b(n, k) = b(n -1,k-1) + (n—1)b(n—-1,k). 


Indeed, F,,(z) = (2 +n — 1)F,-1(z), so we have 


S "b(n, k)z* = So b(n, k)z**? + (n—1) So b(n = 1, k)z* 


n n n—1 
k=0 k= 


1 k=0 
by definition. Taking the coefficient of z* on both sides of the last equation 
we get what was to be proved. 


Theorem 3.20 has a plethora of applications which we will keep encountering 
in this book. For now, just note that setting z = —1 leads to the identity (for 
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n > 2) 


=>  e(n,k)(-1)*. 
k 

That is, there are as many permutations of length n with an odd number 
of cycles as there are with an even number of cycles. See Exercise 74 for a 
similar application. 

Note that Theorem 3.20 immediately implies the following property of the 
signless Stirling numbers of the first kind (a property that is often difficult to 
prove for other sequences). 


COROLLARY 3.21 
For any fixed positive integer n, the sequence {c(n,k)}o<k<n has real zeros 
only. In particular, this sequence is log-concave, and therefore, unimodal. 


We point out that a far-reaching generalization of this result has been 
proved by Francesco Brenti. See Problem Plus 10 for that result. 

By now, you probably know that our next question will be whether there 
is a combinatorial proof of the log-concavity of the sequence {c(n,k)},. The 
answer is in the affirmative, as shown by the following construction due to 
Bruce Sagan [278]. In this proof, we modify our cycle notation a little bit by 
writing our cycles with their smallest entry first. This will be called reverse 
cycle notation. 

In the construction of Sagan, and in solving some of the exercises of this 
chapter, the notion of gap positions turns out to be useful. An n-permutation 
written in cycle notation has n +1 gap positions, one after each entry p; (this 
gap position is in the same cycle as p;), and one at the end of the permutation, 
in a separate cycle. For instance, if p = (1)(23), then inserting 4 into the first, 
second, third, and fourth gap position of p, we get the permutations (14)(23), 
(1)(243), (1)(234), and (1)(23)(4). Note that the reverse canonical form of p 
is preserved, no matter where we insert 4. This is why we chose this kind of 
cycle notation. 

Let P(n,k) denote the set of n-permutations with k cycles. Define a map 
®: P(njk —1) x P(n,k +1) > P(n,k) x P(n,&) as follows. 

Let p € P(n,m). Let py be the permutation obtained from p by removing 
all entries larger than 7 from the cycle decomposition of p. 


Example 3.22 
Let p = (124)(35), then py) = (1), pia) = (12), prs) = (12) (3), pyay = (124)(3), 
and pis) = (124)(35). 


Note that if pi;, has ¢ cycles, then pi;41) has either ¢ or t + 1 cycles. Now 
let (p,q) € P(n,k —1) x P(n,k +1), and look at the sequences {py bi<i<n 
and {4q;) }i<i<n- Let j be the largest index for which p,;) has one cycle less 
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than q,;). Such an index must exist as the difference between the number of 
cycles of q;) and py; changes by at most one at each step, starts at 0 (when 
i = 1), and ends in 2 (when i = n). 

Swap the elements of the pair (p(j),q/j)), to get the pair (qj), pyj)). Now 
insert 7 + 1 into the same gap position of qj) in which it would have to 
be inserted if we wanted to expand p,j) into pyj41). Similarly, insert 7 + 1 
into the same gap position of pj) in which it would have to be inserted if 
we wanted to expand qj) into qij41). Call the new pair of permutations 


obtained this way (q j+1)? Di j+1) ). Then continue with analogous steps. That 


is, construct (Qj4a+1)) Piyj+att)) from (5-40) Pij-+a)) by inserting 7 +a+1 
into the same gap position of qi fo (resp. Pi j a5) that we would use if we 
had to expand pyj+4a) (resp. q(j+a)) into pyj4a41) (TeSP. q(j+a41)). At the end 
of the procedure, we define ®(p, q) = (dn) Pliny): 


Example 3.23 
Let n = 6 and k = 2, and let p = (125463), and q = (13)(24)(56). It is then 
clear that j = 4, and py4y = (1243) and q4) = (13)(24). After swapping, we 
get the pair 

(q(4), P(ay) = ((18) (24), (1243). 

To make further computations easier, we note that pis) = (12543) and 
qX5) = (13)(24)(5). 

The entry 5 would have to be inserted into the second gap position of pi4) 
to get pis), and into the fifth gap position of q(4) to get qi5). So we insert 5 
into the second gap position of q 4) and into the fifth gap position of Di 4) We 
obtain 

(%s5) Psy) = ((135)(24), (1243)(5)). 


Finally, the entry 6 would have to be inserted into the fourth gap position of 
P(5) to get pie) = p, and into the fifth gap position of q(5) to get q6. = q. So 
we insert 6 into the fourth gap position of qi 5) and into the fifth gap position 
of Dis) We end up with 


®(p,q) = (4%) Pie) = ((135) (264), (1243) (56). 


THEOREM 3.24 
The map ® defined above is an injection that maps P(n,k — 1) x P(n,k +1) 
into P(n,k) x P(n,k). 


PROOF _ First, ® indeed maps into P(n,k) x P(n,k). This is because 
passing from pi;. to pyi41) a new cycle is formed if and only if 7 + 1 is inserted 
into the last gap position. By the definition of 7, this happens one more time 
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n=0 1 

n=1 0 1 

n=2 0 1 1 

n=3 0 2 3 1 

n=4 0 6 11 6 1 
n=5 0 24 50 35 10 1 


FIGURE 3.5 
The values of c(n,k) for n < 5. The NE-SW diagonals contain the values for 
fixed k. Row n starts with c(n, 0). 


from qj) tO qin) than from pyj) to p(n). After swapping, this will precisely 
compensate for the extra cycle that we created when passing from q,1) to q(3), 
in comparison to the segment from p,1) to pj). 

To see that ® is an injection, note that if (Gny»Piny) is in the image of 
®, its only preimage is easy to reconstruct. Indeed, remove n from both 
permutations, then remove n — 1 from both permutations, and so on, and 
stop as soon as the first permutation has exactly one more cycle than the 
second. This provides the index 7, and the preimage of (ny Pl ) is then 


n 


obtained by reversing the algorithm used in constructing ®. 


See Figure 3.5 for the values of c(n,k) forO<n <5. 

As there seems to be no symmetry in any row of this diagram, it is natural 
to ask where the maximum of each row is. That is, for fixed n, what is the 
value (or values) of k for which c(n,k) is maximal. Recall that by Corollary 
3.21 and Proposition 1.33, there are either one or two such values of k. (In 
the latter case, the two values of & must be consecutive.) We will call these 
values of k the peak of the sequence. 

Surprisingly, our main tool in answering this question comes from the real 
zeros property of the numbers c(n,k) as proved in Corollary 3.21. This en- 
ables us to use the following powerful, and not quite well-known, theorem of 
Darroch. 


THEOREM 3.25 

Let A(z) = Yy_9 an2* be a polynomial that has real roots only that satisfies 
A(1) > 0. Let m be an index so that ay, < dm for all k, that is, dm is maximal 
among the coefficients of A(z). Let 4 = A’(1)/A(1). Then 


| —m| <1. 


See [130] for the proof of this spectacular result, as well as for further details. 
For certain values of n, an even more precise result is known. 
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Theorem 3.25 is remarkable for the following reason. Let us assume that 
A(z) is the generating polynomial of certain objects according to some pa- 
rameter r. For example, A(z) could be the generating polynomial of n- 
permutations according to their number of cycles, or the generating function 
of all houses in a given town according to their prices. Then 


-_ ‘3 kar 
ig A 


is simply the average value of r over the set of all eligible objects. Indeed, 
there are ax objects for which the value of r is k. On the other hand, m is 
the value that r takes most frequently. 

In general, 44 and m may not be directly connected. For example, it could 
certainly happen that the value of r is never taken by any one object. It could 
even happen that the average value of r is far away from its specific values. 
However, Theorem 3.25 shows that if the generating polynomial A(z) has real 
roots only, then the average value and the most frequent value are less than 
1 apart. 

Let us return to the task of determining the index & for which c(n,k) is 
maximal, while n is kept fixed. By Theorem 3.25, it suffices to compute p, 
and we need to check at most two values of c(n,k) to find the peak of the 
sequence {c(n, k)},%. We say peak, not peaks, because Erdés [165] proved that 
this sequence has a unique peak if n > 3. 

Let A(z) = Tyo e(n, k)z*. All we need in order to apply Theorem 3.25 is to 
compute A’(1)/A(1), that is, the average number of cycles in n-permutations. 
This is the content of the next lemma. 


p= A'(1)/A(L) 


LEMMA 3.26 
The average number of cycles of a randomly chosen n-permutation is 


H(n) =e. 


Lemma 3.26 has several proofs, some of which have far-reaching generaliza- 
tions. For now, we provide an elementary proof. By the end of this chapter, 
the reader will have the techniques to prove the lemma in two additional ways, 
and she will be asked to do so in Exercises 65 and 66. 


PROOF ~~ We prove the lemma by induction on n, the initial case of n = 1 
being trivial. Let us assume that the statement is true for n, and take a 
permutation p on [n]. Now insert n + 1 into any of the n+ 1 gap positions of 
p. This will create a new cycle if and only if n + 1 was inserted into the last 
gap position, that is, in 1/(m+ 1) of all cases. Thus the average number of 
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n=0 il 

n=1 0 1 

n=2 0 =] 1 

n=3 0 2 - 3 1 
n=4 QO -6 ll -6 il 


n=5 0 24 = -50 35-10 1 


FIGURE 3.6 
The values of s(n, k) for n < 5. The NE-SW diagonals contain the values for 
fixed k. Row n starts with s(n, 0). 


cycles in a randomly selected (n + 1)-permutation is 


n + 1 
n+l 


(H(n) +1) = H(n) + —~ 


as claimed. | 


Therefore, we have just shown that for any fixed n. we can locate (up to 
distance 1) the index & for which the Stirling number c(n,k) is maximal. 


THEOREM 3.27 
Let n > 3 be a fixed positive integer. Then the unique value of k for which 
c(n,k) ts maximal is within distance 1 of H(n). 


The time has come to explain the adjective “signless” in Definition 3.17. 


DEFINITION 3.28 The Stirling numbers of the first kind are defined 
by s(n, k) = (—1)"-*e(n, k). 


The values of these numbers for n < 5 are shown in Figure 3.6. 


COROLLARY 3.29 
For all positive integers n, we have 


2(z—1)---(2g-n4+1)= S> s(n, k)z*. 
In other words, s(n,k) is the coefficient of z® in z(z—-1)++-(z-—n+1) = (z)n. 


PROOF _ Substitute —z for z in formula (3.3), proved in Theorem 3.20. 
Then multiply both sides by (—1)”. 
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The motivation of assigning signs to Stirling numbers of the first kind is 
explained by the following theorem, which also shows the close connection 
between the numbers S(n,k) and s(n,k), justifying their similar names. 


THEOREM 3.30 

Let S be the infinite lower triangular matriz whose rows and columns are 
indeted by N, and whose entries are given by S;,; = S(i,j). Let s be defined 
similarly, with s;,; = s(i,j). Then Ss=sS =I. 


PROOF _ Both A = {1,2,27,2°,---} and B = {1,z, (x)a, (x)3,---} are 
bases of the vector space R[z] of all polynomials with real coefficients. Here 
()m = «(a@—1)---(a—m+1). Corollary 3.29 shows that the numbers s(n, k) 
are the coordinates of the elements of B in basis A, while Exercise 21 shows 
that the numbers S(n, k) are the coordinates of the elements of A in basis B. 
In other words, s is the transition matrix from A to B, and S is the transition 
matrix from B to A. Therefore, they must be inverses of each other. 


We mention that an explicit formula for the numbers s(n, k) has been known 
since 1852 [281]. It is significantly more complicated than the corresponding 
Lemma (1.17) for the Stirling numbers of the second kind. It states that 


s(n,k)= caret Garen Eco k+h,h) 


0<h<n—k 


“SCE 


0<i<h<n—k 


This formula is called Schloémilch’s formula, and we postpone its proof until 
the end of the chapter. The reason for this is that in the proof of that formula, 
we will use the generating functions of the numbers s(n, k), which we are going 
to compute shortly. 


3.2.5 Generating Functions for Stirling Numbers 


Proving recurrence relations for Stirling numbers of the first kind combinato- 
rially is not quite as easy as it is for Stirling numbers of the second kind. The 
additional degree of difficulty comes from the fact that the numbers s(n, k) are 
not always positive. Therefore, in many cases, an argument using generating 
functions turns out to be simpler. This is the subject of the present section. 

Define the following double generating function for Stirling numbers of the 
first kind. 
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f(z,u) = S- es s(n, Rak. 


n>0 k=0 


PROPOSITION 3.31 
The equality 
f(z,u) = (1+ 4)? 


holds. 


PROOF The coefficient of u” on the right-hand side is (7) by the Bino- 
mial theorem. On the left-hand side, it is 


Teen ke! Gn a 


n! n! 


and the statement is proved. We have used Corollary 3.29 in the above equal- 
ity. 


Sometimes an alternative form of f(z, u) is easier to use. 


PROPOSITION 3.32 
The equality 


holds. 


PROOF Immediate from Corollary 3.29. ff 


The nice, compact form of the double generating function f(z, u) results in 
a plethora of recurrence relations on the numbers s(n,k). These recurrence 
relations are sometimes classified into three sets, triangular, vertical, and hor- 
izontal recurrences. These names are based on the arrays of the numbers 
s(n, k) involved. 

A triangular recurrence is readily obtained from Lemma 3.19. It is 


s(n, k) = s(n—1,k —1) -— (n— 1)s(n —-1,k), (3.4) 


where n > 1. 
Now we are going to use our generating functions to obtain a vertical re- 
currence. 
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LEMMA _ 3.33 
For all fixed positive integers k and n, the equality 


holds. 


PROOF _ It is clear from Proposition 3.31 that 


Of (z,u) 
Oz 


The coefficient of z*~1u"/n! on the left-hand side is ks(n,k). The right-hand 
side equals 


ue uf 
f(z,u) S0(-1 Ss Soa s(i, j)z7— Dea 


g>1 i>0 j=0 ! é>1 


= f(z,u)In(14 z). 


Therefore, the coefficient of z*~1u"/n! on the right-hand side is 


n-1 


1 1 

n—l-1 

nt S> (-1) oe mee Air 
f=k-1 

proving our claim. | 


Sometimes a univariate generating function is sufficient. The “horizontal” 
generating function of the numbers s(n, k) was given in Corollary 3.29. Their 
“vertical” generating function is computed below. 


LEMMA 3.34 
For any fixed k, the exponential generating function of the Stirling numbers 
of the first kind is given by 


= u” In(1 + u)]* 
Jpn? Ee 


PROOF ~ Changing the order of summation in f(z, u), we get 
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The proof now follows by equating the coefficients of z* in two forms of f(z, wv), 
in the one above, and in 


f(z,u) = (14+ u)* = exp[z In(1 4+ w)] 


From here, the analogous result for c(n, k) is a breeze. Note that in the end 
of this chapter, we will see how to deduce this corollary by a more direct and 
more combinatorial argument. 


COROLLARY 3.35 
For any fixed k, the exponential generating function of the signless Stirling 
numbers of the first kind ts 


= uy” —In —u k 
hy(u) = So e(n, k= ae 
n=k : u 


Finally, we are going to prove a horizontal recurrence relation. Interestingly, 
the vertical generating function f,(u) will be used in our proof. 


LEMMA 3.36 
For all positive integers k and n, the equality 


s(n +1,k+1)= ie as (‘") s(n,m) 


holds. 


PROOF Differentiating our generating function f;,(u) (after shifting 
indices by one), we get 


Co 


SS nied ee 
n=k 
n U k; 
= exp(—In(1 + yy Grol 
eo (-1)™[In(1 +.u)J™ = [In(1 + u)]* 
= ae ee 
Ss (prin +0)” 
a5 SS 
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Note that f;,(u) itself is very similar to the last member of the last expres- 
sion. This suggests that we further rearrange the last member as follows. 


s(n +1,k+ 1) = Sy * (| > s(n, m)— 
n! k n! 


Co 


n=k m=k n=mM 
7 pa ae (7) sim) a 
n=k Lm=k 
and the result follows by equating the coefficients of ue, | 


3.2.6 Application: Real Zeros and Probability 


There is a surprisingly strong connection between polynomials having real 
zeros and random independent trials. It is described by the following theorem 
of P. Lévy. 


THEOREM 3.37 

[Lévy’s theorem] Let (ao, @1,°+- ,@n) be a sequence of non-negative real num- 
bers, let A(z) = (y_,) anz*, and let us assume that A(1) > 0. Then the 
following are equivalent. 


(i) The polynomial A(z) is either constant or has real zeros only; 


(ti) The sequence (ag/A(1),a1/A(1),--- ,@n/A(1)) is the distribution of the 
number dy, of successes in n independent trials with probability p; of 
success on the ith trial, for some sequence of probabilities 0 < p; < 1. 
The roots of A(z) are given by —(1— p;i)/p; for i with p; < 0. 


Note that the probability of success at trial i has to be independent of the 
result of all other trials, for all 7. 


Example 3.38 

Let a; = ("). Then A(z) = (1+ 2)”, so A(z) indeed has real zeros only. For 
all 2, let the ith trial be the flipping of a fair coin, and let heads be considered 
success. As the coin is fair, p; = 1/2 for all i. Then ax/A(1) = ({)/2” is 
indeed the probability of having exactly k successes. The roots of A(z) are 
all equal to —(1 — p;)/pi = —1. 


This theorem shows that we can deepen our understanding of a given se- 
quence having real zeros by finding its probabilistic interpretation. We would 
like to find a probabilistic interpretation for the signless Stirling numbers of 
the first kind. That is, we want to find a sequence of n independent trials in 
which the probability of having k successes is c(n,k)/nl. 
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Consider the following sequence of trials. We have one orange ball in a box. 
Let us pick a ball at random, (at the first trial, we have only one choice), then 
let us put this ball back in the box, along with a blue ball. Keep repeating 
this procedure n times. A success is when we draw an orange ball. Let p, be 
the probability of & successes in our n trials. 

Let A; be the event that we choose an orange ball at trial 7. Then P[A,] = 
1, and we are going to compute P[A,]. At the beginning of trial 7, there is 
one orange ball and (j — 1) blue balls in the box, showing that 

1 


1 
MIS TeG= 7 


Consequently, the probability of failure at trial j is P[A;] = ae As our n 
trials are independent, the probability of k successes in n trials is computed 
as 


dy = 5° P[Aj,]P[Ay.]--- P[Aj,]P[A - P[A;,,], 


Pirer| a 


where the sum is taken over the family F of all possible (n—k)-element subsets 
{jk+15jk+25°** 5 Jn} of the set {2,3,--- ,n}. (There is no chance of failure on 
the first trial.) Using the probabilities computed above, this yields 


de =< SUdeaa — G42 — 1) Ga - 1). 
_ F 


Exercise 35 shows that the value of the sum above is c(n,k), implying that 


c(n, k) 


n! 


Pk = . 
Note that while Lévy’s theorem is often a useful tool to prove the real zeros 

property, it is relatively easy to make a mistake while doing so. See Exercise 

27 for a correct application of this method, and see Exercise 63 for a caveat. 


a 


3.3 Cycle Decomposition versus Linear Order 
3.3.1 Transition Lemma 


We have seen two different ways of looking at permutations. One considered 
permutations as linear orders of [n] and denoted them by specifying the order 
P1p2°+*Pn of the n elements. The other considered permutations as elements 
of the symmetric group $,, and denoted them by parenthesized words that 
described the cycles of the permutations. We have not seen, however, too 
many connections between the two different lines of thinking. For instance, we 
have not analyzed the connection between the “visually” similar permutations 
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2417635 and (2)(41)(7635). Fortunately, such a connection exists, and it is 
a powerful tool in several enumeration problems. This is the content of the 
following well-known lemma that is due to Dominique Foata. 


LEMMA 3.39 

[Transition Lemma] Let p be an n-permutation written in canonical cycle 
notation, and let f(p) be the n-permutation written in the one-line notation 
that is obtained from p by omitting all parentheses. Then the map f : Sn > Sx» 
is a bijection. 


Applying this bijection to the example of the paragraph preceding the 
lemma, we have f((2)(41)(7635)) = 2417635. 


PROOF It is clear that f indeed maps into S,,. What we have to show is 
that it has an inverse, that is, for each n-permutation q, there exists exactly 
one p so that f(p) =4¢. 

Let g = q192°::Gn be any n-permutation, and let us look for its possible 
preimages under f. The first cycle of any such permutation p must obviously 
start with q,;. Where could this cycle end? As p is in canonical notation, each 
cycle starts with its largest entry. Therefore, if g; > q,, then q; cannot be part 
of the first cycle of p. So if 7 is the smallest index so that q; > qm, then the 
first cycle of p must end in gj;_1 or earlier. On the other hand, we claim that 
this cycle cannot end earlier than in qg;_1. Indeed, by the definition of 7, we 
have qx < q if 1 < k <j, so if the second cycle started somewhere between 
q and q;, it would start with an entry less than q;. That would contradict 
the canonical property of gq. 

Thus the second cycle of p must start with the leftmost entry q; of q that 
is larger than q;. By the very same argument, the third cycle of q must start 
with the leftmost entry of q larger than g;, and so on. The procedure will 
stop with the cycle that starts with the entry n. This yields a deterministic, 
and always executable, algorithm to find the unique preimage of q under p. 


In our running example, we have q = 2417635. This yields q; = 2. The 
smallest 7 so that q; > q is 7 = 2, so the second cycle will start in the second 
position. Then, the leftmost entry larger than 4 is 7, so the third cycle starts 
with 7, and we get the permutation (2)(41)(7635). 

Taking a second look at our proof, we note that we can explicitly describe 
the entries of gq at which a new cycle is started. These are precisely the 
elements that are larger than everything on their left. This is an important 
notion. 
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DEFINITION 3.40 Let q = q1q2:::dn be a permutation. We say that 
q ts a left-to-right maximum if, for all k <1, the inequality qx < q holds. 


Example 3.41 
The permutation 21546837 has four left-to-right maxima. These are the en- 
tries 2, 5, 6, and 8. 


Left-to-right minima, right-to-left maxima, and right-to-left minima are 
defined accordingly. 


COROLLARY 3.42 
The number of n-permutations with exactly k left-to-right maxima is c(n,k). 


PROOF Bijection f of Lemma 3.39 maps the set of n-permutations 
having exactly k cycles onto this set of permutations. 


3.3.2 Applications of the Transition Lemma 


Lemma 3.39 has several interesting applications to random permutations. We 
will discuss these in Chapter 4. For now, we fulfill an old promise by proving 
Theorem 1.36. For easy reference, that theorem stated that the number of 
n-permutations having exactly k — 1 excedances is A(n,k). Also recall that i 
is an excedance of the permutation p = p,p2:--pPn if pj > i. Similarly, 7 is a 
weak excedance of p if p(t) > i. 


PROOF (Of Theorem 1.36). We show that the bijection f of Lemma 
3.39 maps the set of n-permutations with k weak excedances onto the set of 
n-permutations with k — 1 ascents. 

The proof will be somewhat surprising. The definition of excedances is given 
in terms of permutations as linear orders, so one would not expect to count 
excedances on permutations that are written in cycle notation. However, this 
is precisely what we will do. 

Let m = (pi: ++ Diz) (Pi, 41° ++ Dig) ++ (Pi;_141°+* Pi;) be written in canonical 
cycle notation. 

We now apply f to 7, and count the ascents. We claim that 7 is an ascent 
of f(m), that is, p; < pi4i if and only if i #4 n and p; < r(p;). 

To see the “only if” part, note the following. Unless 7 is a position at the 
end of a cycle, we have m(p;) = pj+1 as each element of a cycle is mapped 
into the one immediately on its right, except for the last one. Therefore, 
pi < pi+1 implies p; < m(p;). When 7 is a position at the end of a cycle, 
then p; < pj+1 always holds because of the canonical property, and so does 
pi < (p;). Indeed, p; is mapped to the leftmost entry (in the canonical cycle 
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notation) in its cycle, and that is always larger than p; (or equal to p;, if the 
cycle containing p; is a 1-cycle). 

The proof of the “if” part is similar. If we have p; < 7(p;), then either 
i is a position not at the end of a cycle, or it is at the end of a cycle. In 
the former case, 7(p;) = pi+1, and therefore the condition p; < m(p;) implies 
pi < pi41. In the latter case, 7(p;) = p; if the cycle containing p; is a 1- 
cycle, and p; < pj+1 if the cycle containing p; is not a 1-cycle, as in that case, 
pi < (pi) < piti1- Indeed, p; is less than the first entry of its cycle, and that 
entry is in turn less than the first entry of the next cycle by the canonical 
property. 

We have shown that the bijection f of Lemma 3.39 turns the weak ex- 
cedances of 7 other than n into ascents of f(a). Finally, Exercise 13 of Chap- 
ter 1 shows that the number of n-permutations with k weak excedances is the 
same as the number of n-permutations with k — 1 excedances, completing the 
proof of our theorem. 


Example 3.43 

Let 7 = (32)(514)(76). Then in the one-line notation, we have 7 = 4325176. 
This permutation has four weak excedances, namely at 1, 2, 4, and 6. On the 
other hand, f(z) = 3251476 has indeed three ascents, at 2, 4, and 5. ] 


Note that f() had its ascents at 2, 4, and 5. This is where 7 has its weak 
excedances (besides the entry mapping into n), if we use the notations of the 
proof, that is, 7 = (32)(514)(76) = (p1p2)(p3paps)(pep7). Indeed, entries 2, 1, 
and 4, that is, the second, fourth and fifth entries, are mapped into a larger 
entry. 

Lemma 3.39 can also be used to prove the following surprising fact. We have 
computed in Lemma 3.26 that the total number of cycles in all n! permutations 
of length n, is n!H(n) = n\(1+4+---++4). On the other hand, it is not hard 
to prove, either as a special case of Exercise 18 or directly, that similarly, 
c(n + 1,2) = n!H(n). In other words, there are as many cycles in all n- 
permutations as there are (n + 1)-permutations with exactly two cycles, if 
n > 1. The fact that these two sets are equinumerous certainly asks for a 
bijective proof, and we are going to provide one. The crucial idea will be that 
omitting n+ 1 from its cycle can split that cycle up into many smaller cycles. 

Recall the map f : S, — S, of the Transition Lemma. This map was 
defined on permutations of length n. With a slight abuse of notation, we will 
also use the notation f for the analogous map defined on any S,,. This is 
because we will use f on a permutation, and then on some of its parts. 

Let 

A = {p € Sn41: p has two cycles} 


and let 
B={(q,C):q€ S, and C is a cycle of q.} 
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Define g: A + B as follows. Set FE to be the cycle of p that does not contain 
the entry n+1. Let D be the other cycle of p, the one that contains n+1. Then 
we can consider D as a permutation, except it is not a permutation of [nl, 
but of some other set S. Apply f to D (that is, omit the parentheses), then 
omit n+ 1 from the resulting permutation of S', to get the new permutation 
D'. Now apply f~+ back to D’ to get a permutation q/ of S— {n +1} that is 
written in canonical cycle notation. Then we define g = Eq’, that is, q is the 
product of the disjoint cycles E and q’. Finally, we set g(p) = (q, E). 


Example 3.44 

Let n = 8, and let p = (4231)(97586). Then B = (4231), and D = (97586). 
Therefore, D’ = 7586, so q’ = (75)(86). This yields q = (4231)(75)(86), so we 
have g(p) = (q, E) = ((4231)(75)(86), (4231)). 


THEOREM 3.45 
The map g: A— B is a bijection. 


PROOF ~~ We prove that g has an inverse. That is, we show that for any 
(q,C) € B, there is exactly one p € A so that g(p) = (¢,C) holds. 

The crucial idea is that we can use n+ 1 to “glue” together all cycles 
except for C’ into one cycle. Let C1, C2,--- ,C be the cycles of g in canon- 
ical order, with C; = C. To get the preimage of the pair (q,C) under g, 
omit all parentheses from the string CiC2---Cj_1Cji41---Cx, then prepend 
the obtained permutation with the entry n+ 1, to get the single cycle H. 
Then define p to be product of the two disjoint cycles C and H. This shows 
that (q,C) has a preimage. (Note that if g had only one cycle, then the list 
C1 C2--+Ci_-1Ci41---Cr is empty, and we just create a l-cycle consisting of 
the entry n + 1 only.) 

To see that (q,C) cannot have more than one preimage is straightforward. 
Indeed, a change in either cycle of p clearly results in a change of f(p). | 


Example 3.46 

To find the preimage of the pair (q, C) = ((4231)(75)(86), (4231)) under g, we 
prepend the permutation 7586 by the maximum element 9. This leads to the 
cycle (97586), yielding g~!(q,C) = (4231)(97586) as we expected because of 
Example 3.44. JJ 
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3.4 Permutations with Restricted Cycle Structure 
3.4.1. Exponential Formula 


We start with a very brief introduction to the main enumerative tool of this 
section. Many of the readers may have seen this material in a general combi- 
natorics textbook. Other readers may consult Chapter 8 of [74] or Chapter 3 
of [75] for a longer treatment on an introductory level, or Chapter 5 of [297] for 
a discussion at an advanced level. The book [169] is the most comprehensive 
treatment of the tools we will use. 

Recall that the exponential generating function of the sequence ao, a1, G2,°-° 
of complex numbers is the formal power series 


A(z) = S- ane. 


n>0 


It is not always necessary to require that the a; be complex numbers. In 
general, it suffices to assume that they are elements of a field of characteristic 
Zero. 


LEMMA 3.47 Product Formula 
Let f(n) be the number of ways to build a certain structure on an n-element 
set, and let g(n) be the number of ways to build another kind of structure on 
an n-element set. Let F(z) and G(z) be the exponential generating functions 
of the sequences f(n) and g(n), for n = 0,1,2,---. Finally, let h(n) be the 
number of ways to partition an n-element set into two subsets (which are 
allowed to be empty), then to put a structure of the first kind on the first subset, 
and to put a structure of the second kind on the second subset. Let H(z) be 
the exponential generating function of the sequence h(n), for n = 0,1,2,--- 
Then the identity 

F(z)G(z) = H(z) (3.5) 


holds. 


PROOF The number of ways to split an n-element set into a k-element set 
and its complement, then build the above structures on these sets is obviously 
(2) f(k)g(n — k). Summing over all k, we get that 


nin) = > (2) £0o(0 = W). (3.6) 


k=0 


So our lemma will be proved if we can show that the expression on the right- 
hand side of (3.6) is in fact the coefficient of z”/n! in F(z)G(z). This is simply 
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a question of computation as 


n! nl 
n>0 n>0 
n ” 1 
- 2 d, Fin byt Ham — F) 


We have two remarks about the two subsets into which we have partitioned 
our n-element set. First, the partition is ordered, that is {1,3}, {2,4,5} is 
not the same partition as {2,4,5}, {1,3}. Indeed, in the first case, we build 
a structure of the first kind on the block {1,3}, and in the second case we 
build a structure of the second kind on that block. Second, empty blocks are 
permitted. This is why the generating functions F(z) and G(z) start with a 
constant term that is not necessarily 0. In other words, our two subsets form 
a weak ordered partition of our n-element set. 

Iterated applications of Lemma 3.47 lead to the following. 


COROLLARY 3.48 

Let k < n be positive integers. For i € [k], let fi(n) be the number of ways 
to build a structure of a certain kind i on an n-element set, and let F;(z) be 
the exponential generating function of the sequence f;(n). Finally, let h(n) be 
the number of ways to partition an n-element set into an ordered list of blocks 
(B,, Bo,--- , Br) where the B; are allowed to be empty, then build a structure 
of kindi on B;, for eachi € [k]. If H(z) ts the exponential generating function 
of the sequence h(n), then the identity 


holds. 


Finally, note that it was not particularly important in our proof that the 
numbers f(n) were integers, or that they counted the number of certain struc- 
tures. What was important was the relation between f(n) and h(n). This 
leads to the following generalization. 


THEOREM 3.49 
Let K be a field of characteristic zero, and let fi : N — K be functions, 
1<i<k. Defneh:N->K by 


h(n) = Se fi(/Ai|) fo(|Aal) «>> fe Ae), 
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where the sum ranges over all weak ordered partitions (A, A2,--- , Ax) of [n] 
into k parts. Let Fi(z) and H(z) be the exponential generating functions of 
the sequences fi(n) and h(n). Then the identity 


A(z) = F,(z)Fo(z)-+- Fr(z) 
holds. 


You could ask what the advantage of letting f and h map into any field of 
characteristic zero is, as opposed to just the field of real or complex numbers. 
The answer will become obvious when we discuss cycle indices later in this 
section. For the time being, we just mention that K will often be chosen to 
be a field of rational functions over a field. 

Theorem 3.49 yields an immediate second proof of Corollary 3.35. For easy 
reference, that corollary claimed that the exponential generating function of 
the signless Stirling numbers of the first kind is 


= u” —In —u k 
hu(u) = So e(n, b> = See 
n=k y i 


Indeed, hy(u) is just the exponential generating function of the numbers 
c(n,k). Note that k!c(n, &) is the number of ways to choose an n-permutation 
with k-cycles, then order its set of cycles. However, this is the same as parti- 
tioning [n] into k non-empty subsets (Aj, Ag,--- , Ax), then taking a 1-cycle 
on each of the A;. Therefore, with the notations of Theorem 3.49, we have 
filk) = (k — 1)! for each i, with f(0) = 0. This yields Fi(u) = 7%,“ = 
—In(1 — wu) for each i, and our claim is proved. 

Our main tool in this section, the Exponential formula, will be stated and 
proved shortly, using Theorem 3.49. First, however, we have to take some 
precautions. We want to work with power series of the type exp(F'(x)), where 
F(x) is some formal power series. 

A little thought shows that exp(F(z)) = 1+ F(z) +F(z)?/2+--- is defined 
if and only if F(0) = 0, or in other words, when F' has no constant term. 
Indeed, it is precisely in this case that the coefficient of any z™ in 1+ F(z)+ 
Boe = ee Fay will be a finite sum. Therefore, we will have 
to make sure we only talk about exp(F(z)) when we know that F' has no 
constant term. 

Having settled that, we are ready for the exponential formula. Let P denote 
the set of all positive integers. 


THEOREM 3.50 
[The Exponential Formula] Let K be a field of characteristic zero, and let 
f:P—>K bea function. Defineh: N > K by h(0) = 1, and by 


h(n) = > f(Ail): f(lAal) ++ f(Aml), (3.7) 
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forn > 0, where the sum ranges over all partitions (Aj, A2,-+- ,Am) of [n] 
into any number of parts. Let F(z) and H(z) be the Bmancnial generating 
functions of the sequences f(n) and h(n). Then the identity 


H(z) = exp(F(z)) 


holds. 


In particular, if f(n) is the number of ways one can build a structure of a 
certain kind on an n-element set, then h(n) is the number of ways to take a 
set partition of an n-element set, and then to build a structure of that same 
kind on each of the blocks. 

Note that in contrast with the product formula, here we take a partition of 
our n-element set. This means that the set of blocks is not ordered, that is, 
{1,3}, {2,4,5}, and {2,4,5}, {1,3} are considered identical, and empty blocks 
are not permitted. There is no restriction on the number of the blocks, unlike 
in Lemma 3.47 and in Corollary 3.48. This is why we have to exclude empty 
blocks; otherwise the number of our partitions would be infinite. 


PROOF _ (of Theorem 3.50) Let 


= S~ f(/Ail) fF (Aa): F( An) 


where the sum ranges over all partitions of [n] into k parts, for a fixed k. 

Now that the number of blocks is fixed, we can use Corollary 3.48 (with 
f (0) = 0), keeping in mind that in that corollary, the set of blocks is ordered. 
Recall that [z”]t(z) denotes the coefficient of z” in t(z). We get 


Summing over all k, we get that 


= So hen) = Ole] (F)*) = [e"lexp(F 2), 
k 


k 


which was to be proved. | 


We point out that there is a stronger version of this theorem, the Compo- 
sitional formula, but we will not need that here. Interested readers should 
check Exercise 48. 


Example 3.51 
Let h(0) = 1, and let h(n) be the number of ways to take a set partition of [n] 
and then take a subset of each block. Then the Exponential Formula yields 


H(z) = Cys h(n)= = exp (exp(2z)-1). 0] 
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SOLUTION _ Let f(n) be the number of ways to choose a subset of a 
nonempty set. Then f(n) = 2” for n > 0, and f(0) = 0. Therefore, the 
exponential generating function of the sequence f(n) is F(z) = exp(2z) — 1, 
and the proof follows from the Exponential formula. 


COROLLARY 3.52 
[Exponential formula, permutation version] Let K be a field of characteristic 
0, and let f: P— K be any function. Define a new function h by 


h(n) = S> f (Gil) F(Cal)- ++ F(Cel) (3.8) 


PESn 


where the C; are the cycles of p, and |C;| denotes the length of Ci. Let F(z) 
and H(z) be the exponential generating functions of f andh. Then the equality 


H(2) = exp (0 F(n)=— 


n>1 


holds. 


PROOF = Note the subtle difference between the summation on the right- 
hand side of (3.8) and the summation in (3.7). The former is a summation 
over all n! elements of S,,, the latter is a summation over all set partitions of 
[n]. As a k-element block of a partition can form (k — 1)! different k-cycles, a 
partition with blocks B,, Bz,--- , By gives rise to [ay (|B;|—1)! permutations 
whose cycles are the blocks B;. Therefore, (3.8) can be rearranged as 


h(n) = S) f(| Bal) Bil — YF (Bal) ((Be| — 1)!--- FU Bel) Bel — D!, 


Ben 


where II, denotes the set of all partitions of [n]. 
Then the Exponential formula (applied to (n — 1)!f(n) instead of f(n)) 
implies 
att 
H(z) = exp((n — 1)!- F(z) = exp | )0 f(n)— ], 


n 
n>1 
which was to be proved. | 


The following application of Corollary 3.52 enables us to count permutations 
that consist of cycles of certain lengths only. 
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THEOREM 3.53 
Let C be any set of positive integers, and let gco(n) be the number of permu- 
tations of length n whose cycle lengths are elements of C’. Then the identity 


Go(z) = ¥ se(n)= = exp » *) 


n>0 nec 


holds. 


PROOF Let f(n) = (n—-1)! ifn € C, and let f(n) = 0 otherwise. Then 
f gives the number of ways we can cover a set of n elements by a cycle of an 
acceptable length. Then F(z) = {7% f(n)4, = Nynec =- Our claim then 


follows from the Exponential formula, permutation version. 

An involution is a permutation p so that p~! = p; in other words, p? is 
the identity permutation. It is easy to see that this happens if and only if all 
cycles of p have length 1 or 2. Note that 1-cycles are often called fixed points. 


Example 3.54 
Let G2(z) be the exponential generating function for fixed point-free involu- 
tions. Then the identity Gz(z) = exp(z?/2) holds. 


COROLLARY 3.55 
The number of fixed point-free involutions of length 2n is h(n) = 
1-3--+-(2n—1). 


PROOF _ By Example 3.54, we only have to compute the coefficient go(n) 


of aa in Gg(z) = exp(z?/2). We have 
yen 
exp(z?/2) = X Daal 
so 
g2(n) = Cut fe Binds n=) Sas Ih 


The symbol (2n — 1)!! reads “(2n — 1) semifactorial”, referring to the fact 
that we take the product of every other integer from 1 to 2n — 1. 

Note that G2(z) does not contain terms with odd exponents. This makes 
perfect sense as a fixed point-free involution cannot be of odd length. 


120 Combinatorics of Permutations, Third Edition 


In what follows, we are going to look at some interesting enumerative results 
for some particular choices of the set C. Many of these results could be 
obtained by some clever combinatorial arguments and without the use of 
generating functions. We, however, prefer to present the general technique of 
generating functions, and leave the bijective proofs for the exercises. 

Recall that integrating the identity 1/(1—z) = }7,,.) 2” we get the identity 


—in(l—z) =In((1—2))1) = _ =>) = (3.9) 


n>0 


The following example introduces a widely researched class of permutations. 


Example 3.56 
Let D,, be the number of fixed-point free n- a lee (or derangements). 


n 


Then the identity D(z) = 0,59 Dn= = PH holds. =] 


nl! 


PROOF In this case, we choose C to be the set of all positive integers 
larger than one. Then we have 


D(2) = Ge(2) = exp [= 
= exp(In ((1 — z)~*) — z) saa 


Routine expansion of the last term yields the following formula for the 
number of derangements of [n]. 


COROLLARY 3.57 
Let D,, be the number of derangements of length n. Then 


Dy, =n! we 
i=0 


Note that this formula implies that 
D(n) 1 


lim =- 
noo ni e 


so for n large enough (meaning in this case, n > 1), more than one-third of 
all permutations are derangements. 
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The number of all elements of the symmetric group S;,, whose order is odd 
turns out to have an interesting form. These are permutations with odd cycles 
only. This formula is the content of our next example. 


Example 3.58 
Let C be the set of all odd positive integers. Then the identity Go(z) = ,/+ 


1l-z 
holds. 


PROOF By Theorem 3.53, we need to compute 


23 2 gent 
Gel2) =ew (24 5 454--.) = exp ea 
Now note that taking derivatives yields 
/ 
2n+1 1 1 1 
oP") eves tegtytay 
as 2n+1 ae 1l-z 21—z) 2(142) 


Therefore, by r integration we obtain 


gent 


1 
—— =-In((l-z)')+m(1 


Canes (Sma 2))) +14 ‘)) = (re 


which was to be proved. | 


and so 


COROLLARY 3.59 

For all positive integers n, the number of permutations of length 2n that 
have odd cycles only is ODD(2n) = (1-3-5-++(2n — 1))? = (2n — 1)!!?. 
Similarly, the number of permutations of length 2n+ 1 that have odd cycles 
only is ODD(2n + 1) = (1-3-5-++(2n — 1))?(2n +1) = (2n — 1)!!?(2n + 1). 


This remarkable result has several proofs. A combinatorial one for a more 
general version is given in Exercise 57. 


PROOF Using the result of Example 3.58, all we have to do is find the 
coefficients of 2” /m! in Gco(z) = ,/¢+£. Multiplying both the numerator and 
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the denominator by 1+ z, we get 


Ge(z) = as (3.10) 
By the Binomial theorem, we obtain 
(1—a?)-¥? = S°(-1)™ Coe (3.11) 
m>0 
= 1 AAC) Cm =H) am 6.19) 
_ Cn" — 2m, (3.13) 
m>0 : 


Note that (1 — z?)~1/? has no terms of odd degree. 
This shows that the coefficient gc(2m) of z?”/(2m)! in our generating 
function Go(z) =(l+z)¢ Gm— il 2m is 


m>0 m!-2™ 


2m — 1)! 


while the coefficient go (2m +1) of 2?*!/(2m + 1)! is 
Gee ipl 


a (2m — 1)!!7(2m + 1), 


= (2m — 1), 


proving our claims. | 


As a natural continuation of our last subject, we will now enumerate per- 
mutations that have cycles of even length only. 


Example 3.60 
Let C be the set of all even positive integers. Then the identity Go(x) = 


\/te holds. =] 


PROOF Theorem 3.53 implies 


yen 


Go(z) = exp On, 
n>1 y 


(3.14) 


Note that the argument of the exponential function on the right hand side 
is very similar to (3.9), with z? playing the role of z. Therefore, (3.14) implies 


Go(z) = exp (Sona = Ay) Ey ae 


as claimed. | 
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COROLLARY 3.61 
For all positive integers m, the number of (2m)-permutations that have even 
cycles only is EV EN(2m) = (2m — 1)!!?. 


PROOF Using the result of Example 3.60, all we have to do is find the 


coefficients of z?"/(2m)! in ,/—L,. We have computed this power series in 
1l-az 


(3.11), and saw that the coefficient of z?”"/(2m)! is indeed (2m—1)!!?. fl 


This leaves us with the intriguing observation that ODD(2m) = EVEN(2m) 
for all positive integers m. This fact asks for a bijective proof. One is given 
in Exercise 55. 

In the above examples, we have had conditions on the cycle lengths based 
on their parity. One could just as well compute the generating function for 
the number of n-permutations with no cycle lengths divisible by k, or all cycle 
lengths divisible by k. See Exercises 52 and 57 for generating function proofs, 
and Problem Plus 4 for combinatorial proofs. 


3.4.2 Cycle Index and Its Applications 


We would like to refine our permutation counting techniques. Theorem 3.53 
allows us to count permutations with a given set of cycle lengths. However, if 
we want to count permutations that have, say, an even number of fixed points, 
or an odd number of 2-cycles, then we cannot apply Theorem 3.53 directly. 

In order to be able to handle complex situations like these, we introduce 
extra variables. Let p be an n-permutation that has a; cycles of length 7. We 
then associate the monomial II_,t/* to p. For example, if p = (312)(54)(6), 
then we associate the monomial titgt3 to p, whereas if p = (21)(3)(54), then 
we associate the monomial t1¢3 to p. 

Summing all these monomials over all n-permutations, we get the aug- 
mented cycle index Z(S;,) of the symmetric group Sy. 

That is 


ZS.) = ZS a) tists?) = ae 
PESn 


where a; is the number of 7-cycles of p. We note that the cycle inder Z(S,) 
of Si, is defined by Z(S,) = 4Z(Sn). 


nl 


Example 3.62 
For n = 3, the equality 


Z(Sn) = t? + 3tyte + 2t3 


holds. 
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The reason the augmented cycle index is a useful tool for the enumeration 
of permutations is the following observation. 


PROPOSITION 3.63 
Let Z(S;,) be defined as above. Then the equality 


Y 2(50) 5 = exp t= 


n>0 i> 


holds. 


PROOF _ Let K be the field of all rational functions in variables t1, t2,--+ , tn 
over Q. Define f: P > K by f(n) = tn. Then 


Z(Sn) = D> FUCA FUC2l) --- FUCEI), 


PESn 


where the C; are the cycles of p. Our claim is then immediate from Corollary 
3.52 (the permutation version of the Exponential formula). 


You may wonder how introducing n new variables and obtaining one new 
equation will help us. As you will see from the upcoming examples, the 
strength of Proposition 3.63 is its generality; we can choose any values for the 
t;, and the equation will still hold. 


Example 3.64 
Let g(n) be the number of n-permutations that have an even number of fixed 
points, and let G(z) = 30,3) 9(n)z"/n!. Then the identity 
exp(—2) 
G(z) = cosh z - ———— 
(z) = cosh z Te 


holds. 


Before we start our proof, we remind the reader of the identity cosh(z) = 


(expz + exp(—z))/2 = donso aor In other words, coshz is obtained by 


omitting the odd terms from exp z. 


PROOF The crucial observation is that 


FES NA rT eA Se See) 
2 
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Applying Proposition 3.63 we get 


n a 


YL sint= = expt) expt) -exp | 30 = | = cosn(2): exp(=2) 


n>0 i>2 


In the last step, we used the result of Example 3.56. | 


Example 3.65 
Let h(n) be the number of n-permutations that have an odd number of 
2-cycles and no fixed points. Then 


Ha): h(n) = sinh (=) -exp | 5° a 


i>3 


PROOF Note that 


2 


The claim then follows just as in the previous example. | 


Let p be an n-permutation. We say that p has a square root if there exists 
another n-permutation q for which gq? = p. Note that p can have many 
square roots. For instance, all involutions are square roots of the identity 
permutation. Whether p has a square root or not is easy to tell from its cycle 
decomposition. 


LEMMA 3.66 

The permutation p has a square root if and only if its unique decomposition 
into the product of distinct cycles contains an even number of cycles of each 
even length. 


Example 3.67 

The permutation (21)(64)(753) has a square root, as it has two 2-cycles, and 
zero cycles of any other even length. The permutation (234 does not have 
a square root, as it has an odd number (one) of 4-cycles. 


PROOF (of Lemma 3.66) Let us assume that p = r?. When we take the 
square of r, the odd cycles of r remain odd cycles. The even cycles of r will 


126 Combinatorics of Permutations, Third Edition 


split into two cycles of the same length. Therefore, the even cycles of r? come 
in pairs, proving the “only if” part of our assertion. 

In order to prove that the condition is sufficient, let us assume that p has 
even cycles (a,---ap,) and (b,---bp;). These even cycles can be obtained by 
taking the square of the (2h)-cycle (ab a2b2a3 ---anbp,). Odd cycles of p, such 
as (didgds5---djd2d4---dj—1) can be obtained as the square of (d)d2---d;). 
So we can find roots for all cycles of p this way, then we can take the product 
of the obtained cycles to be the permutation r. 


As we characterized permutations having square roots by their cycle lengths, 
we can use Theorem 3.53 to find the exponential generating function for their 
numbers. 


THEOREM 3.68 
Let SQn be the number of n-permutations that have a square root, and let 
SOG) =>, SQn yn Then we have 


n=0 
l+z a 
= h—. 
SQ(z) =4/ = Il cosh > 


PROOF Recall that our permutations must have an even number of 
cycles of each even length. Therefore, repeated application of the method 
seen in the proof of Example 3.64 yields 


SQ(z) = (exp z) - (cosh(z?/2)) - (exp 23/3) - (cosh(z*/4))---. (3.15) 


Our claim is then proved recalling that we have already computed the power 
series exp (2 + x +-:- -) in Example 3.58. | 


COROLLARY 3.69 
For all positive integers n, the equality SQon: (2n +1) = SQon41 holds. 


This is an interesting result. It means that when we pass from 2n to 2n+1, 
the number of permutations with square roots grows just as fast as the number 
of all permutations. 


PROOF It suffices to show that the coefficients of 2?” and z?"*! in 
SQ(z) are identical. As [],,, cosh(z7‘/(27)) does not contain terms with odd 


1+z 
1-z 


1l+z Zz 
(a =) ODD(n)—. 


n>1 


exponents, it suffices to show this for 
in Example 3.58 that 


. Recall again that we have seen 
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Corollary 3.59 then shows that ODD(2n + 1) = (2n + 1)ODD(2n), which is 
just what we needed. 


So the probability that a randomly chosen 2n-permutation has a square 
root is equal to the probability that a randomly chosen (2n + 1)-permutation 
has a square root. This nice identity certainly asks for a combinatorial proof. 
One is given in Exercise 61. Not surprisingly, it builds on the combinatorial 
proof of the equality ODD(2n + 1) = (2n + L)JODD(2n). 


3.4.2.1 Multivariate Generating Functions and the Exponential 
Formula 


So far, our applications of the Exponential Formula involved univariate gener- 
ating functions, or the cycle index. Sometimes, it is simpler to use a bivariate 
generating function directly. 


Example 3.70 
Let h(n,k) be the number of n-permutations with & fixed points, and let 
H(z,u) = Vaso eao h(n, k)2-u®. Find an explicit formula for H(z, u). 


SOLUTION _ Set f(n) = (n—1)! for n > 2, and f(1) = y. Then note 
that F(z) = zu+ 0,55 2"/n, and use the exponential formula to get 


A(z, u) = exp(F(z)) 


alt 
= exp zu+ > — 
n 


n>2 
= exp(In(1/(1— z)) + zu— z) 
23 ez(u-1) 
— la 


If we want to know the total number of fixed points in all permutations 
of length n, then it helps if we notice that that number is just >>, kh(n,k), 
which is in turn the coefficient of z”/n! in 


OH (z,u) “ ee os 
Aes lu=1= S- S_ a(n, k)— hut Pea 
n>0 k=1 
It is easy to compute that 


OH(z,u) _ Pasi 
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so setting u = 1, we get the univariate generating function h(z) = }>, 5, 2”. 
This shows that the total number of fixed points in all permutations of length 
n is n!; therefore, the average number of fixed points in such permutations is 
1. This is a well-known fact that can be proved in many ways, and we will 
return to it in Chapter 6. For now, let us mention that h(z) is called the 
cumulative generating function of all cycles. 

If we want to know the difference d, between the number of n-permutations 
with an even number of fixed points and the number of n-permutations with 
an odd number of fixed points, then it helps to note that that difference is 


just 
n 


dy = ¥>(—1)*h(n, k) = [2”/n!] H(z, -1). 


k=0 
Then Example 3.70 implies 


dy = [2"/nl] <— 
n —2)% 


Similarly, if C(z,u) = sae, eae k)=u* is the bivariate generating 
function for permutations according to their length and their number of cycles, 
then we can prove the following interesting-looking formula. 


Example 3.71 
The identity C(z, u) = (1 — z)~” holds. 


SOLUTION _ Set f(n) = (n— 1)!y for all positive integers n. Then the 
Exponential formula yields 


C(z,u) = exp(F(z)) 


exp “= 


n>2 
= exp(uIn(1/(1 — z)) 
= (1-2). 
| 
Note that in the special case of u = —1, we get the generating function 


1 — z, meaning that if n > 2, then there are as many n-permutations with 
an even number of cycles as there are permutations with an odd number of 
cycles. 
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The reader is invited to find proofs of Proposition 3.31 and Corollary 3.35 
using the result of the preceding example. 


3.4.2.2. Proving the Formula of Schlomilch 


We return to the task of proving the formula of Schlomilch. This is a some- 
what complicated computational argument. First, and this is interesting, we 
will need the Lagrange Inversion Formula. It is remarkable that as simple a 
problem as the one at hand calls for this advanced technique. 

Let f and g be two formal power series in R[[z]]. We say that g is the 
(compositional) inverse of f if f(g(z)) = z holds. One can prove that in that 
case, g(f(z)) = z also holds. It is also straightforward to check that if f has 
an inverse, then this inverse is unique. We will denote the inverse of f by 
f‘-). Finally, we point out that f has an inverse if and only if f(0) = 0 and 
the coefficient of z in f is non-zero. 

Recall that if f € R[[z]] is a formal power series, then [z”]f denotes the 
coefficient of z” in f. 


LEMMA 3.72 
[Lagrange Inversion Formula] Let f € R[z]], and let k and n be positive 
integers satisfying 1<k<n. Then the equality 


em(poyt = Epny (2) (3.16) 


n z 


holds. 


The Lagrange Inversion Formula has several interesting proofs, among which 
we can find combinatorial, algebraic, and analytic arguments. See [297] for 
one proof of each kind. The original argument of Lagrange actually proves a 
stronger statement in that & and n are only required to be integers. Gilbert 
Labelle et al. found combinatorial proofs for various versions of the Lagrange 
Inversion formula, including multivariate generalizations. See [86], [192] and 
[241] for these results. 

Now we are in a position to state and prove the formula of Schlomilch. 


THEOREM 3.73 
[281] For all positive integers k and n satisfying k <n, the identity 


sun = a(R T(r ste kta) 


1<j<n—k 


ae) Ce 


1<i<j<n-k 


holds. 
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PROOF First note that the result of Lemma 3.34 provides a formal 
power series whose coefficients are very close to the numbers s(n, k). Indeed, 


* s(n, k) = [u™](In(1 + u))*. 


Then we can apply the Lagrange Inversion formula to the power series f(w) = 
(exp u) — 1 and f(-)(u) = In(1+.u). We get 


* s(n, k) = * ft) a - ‘ (3.17) 


U 


In order to proceed further, we need the following technical lemma. 


LEMMA 3.74 
Let r be any complex number, and let G(u) = ar On be a formal power 


series with go = 1. Define the polynomials P&(u) by the equation 


= (run 
m>1 
In other words, P&) is the coefficient of u™/m in (G(u))". Then the identities 


+r\< - 1 m . 
Pon =r(™ ) -1) (™) re 3.18 
9 =e") ay (™) Pe (3.18) 


J=1 


hold. 


See page 142 of [123] for a proof of this lemma. 


_ (expu)—1 
_ u 


Now let us apply the lemma with G(u) ,r=n,andm=n-—k. 


Using these values, (3.17) implies 


maton ol) Ecos) (aS )) 


The last step is certainly counterintuitive. With significant effort, we have 
made a simple expression much more complicated. The reason for this is 
that now we can recognize S(n — k + j,7) in the last term. Indeed, a simple 
application of the Exponential Formula (see Exercise 29) shows that 


= u” 1 7 k 
2h a 


Therefore, 


? 


no (CO-1\ 5 _ jIS(n— +5, 5) 
le ()s ~ (nk + 3)! 
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and the first equality of the formula of Schlomilch is proved by routine can- 
cellations. One then obtains the second equality by substituting the exact 
formula for S(n —k + j,7) given in Lemma 1.17. 


a 


Exercises 


ae 
2: 


10. 


11. 


12. 


(—) Prove that Alt, has n!/2 elements for n > 2. 
(—) Find a subgroup of S;, that has (n — 1)! elements. 
(—) For p € Sy, prove that p and p~! have the same number of inversions. 


For p,q € Sn, prove that the number of fixed points of pq and qp is the 
same. Try to find a solution using linear algebra. 


(—) Find a surjective homomorphism f : S, > S9. 


(Basic knowledge of Linear Algebra required.) It is routine to check 
that the map f(p) = A, mapping each element of S, to a permutation 
matrix is a homomorphism. Is this homomorphism irreducible? That 
is, is there a nonzero n-dimensional vector v (say, with real coefficients) 
for which A,(v) = v for all p € S,,? 


(—) Let n > 3. Let us assume that we know that f € S;, is such that 
fg =f for allg € S;,. What can f be? 


(—) Let n be an even positive integer. Prove that at least half of all 
permutations of length n contain a cycle of length at least n/2. 


Let L(n,k) be the number of n-permutations whose longest cycle is of 
length k. Let us assume that 3 <k <n. Find a formula for L(n,k). 


Let L(n,k) be defined as in the previous exercise. 


(a) Let }<k < 4. Find a formula for L(n, k). 
(b) (+) Now let 1<k <n. Find a formula for L(n, k). 


An adjacent transposition is a transposition interchanging two consec- 
utive entries in a permutation. Prove that any element of S, can be 
obtained as a product of (not necessarily distinct) adjacent transposi- 
tions. 


(—) Let p € S,, be a permutation that can be obtained as a product of 
m transpositions and also as a product of & transpositions. What can 
be said about m+ k? 


17. 
18. 


19. 


20. 


21. 


22. 
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Let n > 1. Find all values of n for which S,, does not have two conjugacy 
classes of the same size. 


. Prove that any even permutation of S,, can be obtained as a product of 


(not necessarily distinct) 3-cycles. 


. Prove formula (3.1). 


. Let T be a tree on vertex set [n], with degree sequence d1,d2,--- , dn. 


Prove that the number of all cyclic permutations of [n] that are gen- 
erated by the transpositions defined by the edges of T is D(T) = 
dy!dq!-+--d,!. 


(—) Find a simple closed formula for c(n,1) and c(n,n — 1). 
Prove, preferably combinatorially, that 


! 1 
nN. 

c(n,k) = 5 pipe 
“rytrete-+reen ee k 


where the r; are positive integers. 


Prove a formula for S(n,k) that is similar to that of the previous exer- 
cise. 


(+) Give a generating function proof of the identity 


s(n +1,k+1) = $0 (-1)?-™(n)n—m * 9(m, k). (3.19) 


Define a;,(n) = S(n + k,n) and by (n) = c(n,n — k). Prove that for any 
fixed positive integer k, both a,(n) and b;(n) are polynomial functions 
of n. What is their degree and leading coefficient? 


Let az(n) and b(n) be defined as in the previous exercise. We have 
just proved that these functions are polynomials of n. We can therefore 
substitute any real number into these polynomials, not just positive 
integers. Prove that for all positive integers k and all integers n, we 
have 


(a) 


(b) 
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23. 


24. 


25. 


26. 


27. 


28. 
29. 


30. 


bl. 


32. 


33. 


Give a combinatorial proof of the identity 


cn+1,k+1)= S¢ (n) N)me(n, mM), 


m=0 
where we set (n)o = 1, and where0 << k<n 


Let co(n, k) be the number of n-permutations with & cycles in which each 
cycle length is at least two. Find a recurrence relation for the numbers 
c2 (n, k). 


Let c3(n, &) be defined in a way analogous to the previous exercise. Find 
a recurrence relation satisfied by the numbers c3(n, k). 


Find a closed form for the generating function 


a So So Sin, k)— 


n>0 k=0 


A box originally contains m white balls. We run a multiple step ex- 
periment as follows. At each step, we draw one ball from the box at 
random. If the drawn ball is white, we put a black ball in the box in- 
stead of that white ball. If the drawn ball is black, we put it back to 
the box. For k < n, compute the probability of drawing k white balls 
in n trials. What is the connection between these probabilities p(k, 7), 
and the Stirling numbers of the second kind? 


Prove that-S(n,k) = Ye SGm— 1k 1k. 


Find a closed form of the exponential generating function g,(u) = 
Spo (ny ka” fal) 


Find a simple combinatorial definition for the numbers R(n, k) so that 
we have 


c(n, k) = S- c(m—1,k-—1)R(n-—m,k). 


k<m<n 


Then find a formula for the numbers R(n, k). 


Let p be a prime number. Prove that c(p,k) is divisible by p unless 
k=p,ork=1. 


(Wilson’s theorem.) Let p be a prime. Prove that 1+ (p—1)! is divisible 
by p. 


Let p be a prime. Prove that S(p,k) is divisible by p, unless k = p or 
k=1. 
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34. 


35. 


36. 


37. 
38. 


39. 


40. 


41. 


42. 
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(a) Prove that 


1122 ad ‘In—k = S(n, k). 
1St1 Sig S++ Stn Sk 


(b) Prove that 


(i1ig-++in_z)’ = S(n,k, 2), 


1<i1 Sig<---<in 4 Sk 


where S(n, k, @) is the ¢-Stirling number of the second kind as de- 
fined in Problem Plus 3 of Chapter 1. 


Prove that 


4112 aan In—k — c(n, k). 
1<t1 <ig<++s<in-p<n-1 


Prove (preferably with a combinatorial argument) that for fixed positive 
integers n and k satisfying k < n, the equality 


n 


Zameen Heo) 


m=k 
holds. 
Give a proof of Corollary 3.57 without the use of generating functions. 


Prove that D,, (the number of derangements of length n) is the integer 
closest to n!/e. 


Which set is larger, the set of all derangements of length n, or the set 
of all n-permutations with exactly one fixed point? By how much? 


Give a combinatorial proof (no generating functions, or recurrence rela- 
tions) for your answer to the previous exercise. 


Let p be an n-permutation, and let aa(p) be the smallest of all the ascents 
of p, if p has any ascents, and let aa(p) = n otherwise. A desarrangement 
is a permutation for which aa(n) is even. For example, 213 and 54312 
are desarrangements. Let J(n) be the number of desarrangements of 
length n. 


(a) Prove that J(n) = nJ(n — 1) + (-1)”. 
(b) Prove that J(n) = Dn. 


Find a combinatorial proof for the identity J(n) = Dn, where J(n) is 
defined in the previous exercise. 
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43 


44, 


45. 


A6. 


AT. 
A8. 


49. 


50. 


. Prove combinatorially that D, = (n — 1)(Dy-1 + Dn_2). 


Prove that for any fixed n, the sequence {S(n, k)}i<k<n is log-concave. 
(Be careful.) 


Prove that 
c(n+1,1) cn +1,2) +--+ cn +1,k) 
det Ch, = det | OVE AN An tA AEM!) | = cai 
c(n+k,1) cn +k,2)--- cn +k,k) 
How many n-permutations have exactly three left-to-right maxima and 


exactly one left-to-right minimum? 
Give a combinatorial proof for the result of Corollary 3.55. 


[The Compositional Formula] Let K be a field of characteristic zero, let 
f:P-K bea function, and let g: N — K be a function satisfying 
g(0) =1. Define h: N > K by 


h(n) = D0 f( Ail) f(Aal) fl Am|)g(m), 


where the sum ranges over all partitions (Ai, A2,-+: ,Am) of [n] into 
any number of parts. Let F(z) (resp. G(z), H(z)) be the exponential 
generating function of the sequence f(n) (resp. g(n),h(n)). Prove that 


exp(H(z)) = G(F(z)). 


Let h(n) be the number of ways in which n people can sit down around 
an unspecified number of numbered tables, without leaving any tables 
empty. Two seating arrangements are considered identical if each per- 
son has the same left neighbor in both of them. Find the exponential 
generating function of the numbers h(n). 


Let h(n) be the number of ways in which n people can sit down around 
an unspecified number of tables arranged in a circle. Two seating ar- 
rangements are considered identical if each person has the same left 
neighbor in both of them, and, the same holds for the tables. 


In order to further explain this second condition, let A,,A2,--- , A, be 
the tables of one circular arrangement, listed counterclockwise. Here, 
each A; is a cycle that is formed by people. Let By, Bo,---,Br be 


another circular arrangement of tables listed counterclockwise. We say 
that these two arrangments are identical if they are circular translates 
of each other, that is, if there is an index j so that A; = Bj, for all 4, 
where i+ 7 is understood modulo k. 


Find the exponential generating function of the numbers h(n). 


53. 


54. 


55. 
56. 
57. 


58. 
59. 


60. 


61. 
62. 


63. 


64. 
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. (-) Find a formula for the number of n-permutations whose third power 


is the identity permutation. 


. Find a formula for the number of permutations of length n in which 


each cycle length is divisible by k. 


Find the exponential generating function A(z) for the numbers a(n) of 
n-permutations that have an even number of cycles of each odd length. 


Find the exponential generating function B(z) for the numbers b(n) of 
partitions of [n] that have an even number of blocks of each even size. 


Prove combinatorially that ODD(2m) = EVEN(2m). 
Find a combinatorial proof for Exercise 52. 


Find a formula for the number of permutations of length n in which no 
cycle length is divisible by k. 


Find a bijective proof of the equality OD D(2n)-(2n+1) = ODD(2n+1). 
Find a bijective proof of the equality 


ODD(2n +1): (2n+1) = ODD(2n + 2). 


(+) 


(a) Prove that the number of all fixed point-free n-permutations with 
iy e ~2 
descent set {i} is equal to ("77). 
(b) What can we say about the number of all n-permutations having 
exactly one fixed point whose descent set is equal to {7}? 


(+) Find a bijective proof of the equality SQo, -(2n + 1) = SQan41. 


Prove, without using generating functions, that SQon41- (2n + 2) > 
SQon+2- 


Exercise 6 of Chapter 1 provided a probabilistic interpretation for the 
Eulerian numbers. Why cannot we use that interpretation to deduce by 
Lévy’s theorem (Theorem 3.37) that the Eulerian polynomials have real 
roots only? Note that the Eulerian polynomials have irrational roots in 
general, while the interpretation of the mentioned exercise would yield 
rational roots. 


Find a combinatorial proof for the fact that 
De < Dn—-1Dn+1, 


ifn > 3. Here D, denotes the number of derangements of length n. 
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65. 


66. 


67. 


68. 


69. 


70. 
71. 


72. 


73. 


Prove Lemma 3.26 (that is, the statement that the average number of 
cycles in an n-permutation is the harmonic number H(n)), by analyzing 
the derivative of both sides of the equation 


n 


Socln, kz" =2(z4+1)---(z+n-1), 
k=0 


which we proved in Theorem 3.20. 


Prove Lemma 3.26 using the Product formula. (Hint: compute the total 
number of cycles in all n-permutations.) 


Let h(n, k) be the number of n-permutations with exactly k fixed points. 
Find a closed formula for the exponential generating function H(z, u) = 


ees dog lh, k)uk =. 


Let C' be a subset of the set of all positive integers. Consider the power 


series “i 
Wo(z) = et (exp (= = 1)) 


Find the class of permutations whose exponential generating function is 
Wo(z). 


Let f(n,2) be the number of ways to choose an i-element subset S of 
[n], and then to choose an involution g on S. Set f(n,0) = 1. Prove, 
by a combinatorial argument, that 57)", f(n,i)(—1)"~* is equal to the 
number of fixed point-free involutions on [n]. 


Solve the previous exercise using generating functions. 


Let p be a permutation of length n. Let us define the increasing tree 
IT(p) of p as follows. The tree [T(p) is a nonplane tree that has vertex 
set {0,1,---,n}, and root 0. The parent of vertex p; is the vertex p; 
that is the closest vertex to p; that precedes p; for which p; < p; holds. 
If there is no such vertex, that is, when p; is a left-to-right minimum, 
then the parent of p; is the root of IT (p). 


Prove that the map p > IT(p) is a bijection from the set of all per- 
mutations of length n to the set of all non-plane trees on vertex set 
{0,1,--- ,} in which each vertex has a larger label than its parent. 


Consider all n! increasing trees on vertex set {0,1,---,n}. What is the 
total number of all leaves of these trees? 


Consider all n! increasing trees on vertex set {0,1,---,n}. What is the 
total number of all vertices in these trees that have at least one leaf as 
a neighbor? 
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74. A group of n tourists enters a restaurant. They sit down around an 
unspecified number of identical tables, then each table orders one of two 
kinds of drinks. In how many ways is this possible? 


75. What is the total number of 2-cycles in all derangements of length n? 


76. Let d(n,k) be the number of derangements of length n that consist 
of k cycles. Find a closed form for the bivariate generating function 


D(z,U) = Vindo neo Un, k)zruk. 


77. Find an explicit formula for 
f(r) = SCitv), 
Pp 


where the sum is taken over all permutations of length n that consist of 
a single n-cycle. 


Problems Plus 
1. Prove combinatorially that D, = nD,-1 + (—1)”. 
2. (a) Prove that SQan+2 < (2n + 2)SQon41- 
(b) For what values of n does equality hold? 


3. Let po(n) be the probability that a randomly selected n-permutation 
has a square root. Exercises 61 and 2 show that p2(n) is a decreasing 
sequence. Show that p2(n) > 0. 


4. Find a combinatorial proof for Exercise 57. 


5. (a) Let A, = >7"_, +. Prove that 


t=1 it 


n 


1 
en +1,3) => («2 1). 


i=1 


(b) Show ha 
- i=1 a i=l ae 


6. Let g > 2 be a prime, and let p,(n) be the probability that a randomly 
selected n-permutation has a gth root. Prove that p(n) = pg(n + 1), 
except when n+ 1 is divisible by gq. 
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7. 


10. 


11. 


Show that 
e(n+1,k+1) 2 nl [n(n + 1) 4+ CJ /k!. 


What can we say about the constant C’ above? 


. Show that 


kr 
S(n,k) me 


. Inthis problem, we are investigating how many inversions an n-permutation 


can have if it has a fixed number k of cycles. Recall that i(p) denotes the 
number of inversions of p, and that c(p) denotes the number of cycles 
of p. Let 


b(n, p) = min{i(p)|p € Sn, c(p) = k}, 
and 
B(n,p) = max{i(p)|p € Sn, e(p) = kf. 
(a) Find a formula for b(n, k). 
(b) Find a formula for B(n, k). 


(c) Find a formula for the number m(n,k) of n-permutations with k- 
cycles for which the minimum is attained, that is, that have b(n, k) 
inversions. 

(d) Find a formula for the number M(n,k) of n-permutations with 
k-cycles for which the maximum is attained, that is, that have 
B(n, k) inversions. 


An r-derangement is a permutation in which each cycle length is larger 
than r. So a 0-derangement is a permutation, while a 1-derangement is 


a derangement. Let 
Gn r(z) = 5 er), 
Pp 


where the sum is taken over the set D,(n) of all r-derangements of 
length n. Prove that for any fixed n and r, the polynomial G,,,,(z) has 
real zeros only. 


(a) Let D be a conjugacy class of S,. Prove that the polynomial 


Bp(2) = 0, 


pED 
where exc(p) denotes the number of excedances of p, has real zeros 
only. 


(b) It follows from part [(a)] that the polynomial Ep(z) = D7 ,cp gence) 
is unimodal. Where does it take its maximal value? 
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(c) Now let Dz,-(n) be the set of all n-permutations that have exactly 
k cycles, and each of these cycles is longer than r. Prove that the 
polynomial 

Ep,k.rn(2Z) = ye zerc(p) 
pe Dx, + (n) 
is symmetric and unimodal. 


(d) Prove that the polynomial 


Epm@)= dp 2c) 


peD,(n) 


is symmetric and unimodal. Here D,(n) denotes the set of all 
r-derangements as defined in the previous Problem Plus. 


Exercises 41 and 42 defined desarrangements, and discussed some of 
their close connections to derangements. This problem takes that ap- 
proach further. 


Let T € [n — 1]. Prove the following statement: The number of de- 
rangements of length n having descent set T is equal to the number of 
desarrangements of length n whose inverse has descent set T’. 


It follows from the solution of Exercise 22 that there exist integers By, 
with 7 € [k], such that 


Saul zn — Liter Base! ee 


ql = z)ekt1 y 


and 


2k 
y by (n)z” = ijk Bari 12" 


(1 = z)aktl 7 


where eee Bri = (2k —1)!!. Prove that the numbers B;,; are al- 
ways non-negative by finding a set of permutations that these numbers 
enumerate. 


Let S be any proper subset of [n — 1]. Prove that the number of fixed 
point-free n-permutations with descent set S is equal to the number of 
n-permutations with one fixed point having descent set S. 


Let S be any subset of [n — 1]. Prove that there are as many involutions 
on [n] with descent set S as involutions on [n] with descent set [y—1]—S. 


Let prj~ be the number of n-permutations in which every cycle has less 
than k elements. Prove that for any fixed k, the sequence (pn,n)x>2 is 
log-convex, that is, 


2 
Prik S Pn-1,kPn+1,k- 
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17. 


18. 


19. 


Generalize Exercise 41 in the following way. Let F;,, be the number of 
n-permutations with exactly k fixed points. Let E,,, be the number of 
n-permutations p1p2--+ Pp in which p)p2--- px is an increasing sequence, 
and peyiPk+2°'*Pn is a desarrangement. A little thought shows that 
for each n-permutation, there is exactly one non-negative k with this 
property. Prove that Fr, = En,p. 


Let fix(p) be the number of fixed points of the permutations p. Let 
pl be the multiset-permutation obtained from p by replacing each 
fixed point of p by 0, and let dez(p) be the number of descents of p\!. 
Prove that the three-variable statistics (fix,eaxc,d) and (fix, exc, dez) 
are equidistributed. In other words, prove that there is a bijection ¢ on 
the set of all n-permutations so that p and ¢(p) have the same number 
of fixed points, the same number of excedances, and d(p) = dez(¢(p)). 


Let J be a subset of [n — 1]. 


(a) Let p be an n-permutation, and let us assume that D(p) = J. At 
most how many fixed points can p have? 

(b) Let F,(J) be the set of n-permutations with descent set J and 
exactly n — |J| fixed points. Let G(J) be the set of derangements 
q on J so that whenever both 7 and i+ 1 are elements of J, the 
inequality q(t) > q(t +1) holds. Prove that then 


S- yene(p) — S- yeue(q) | 


peF, (J) qeG(J) 


(a) What is the largest possible number of fixed points in an alternating 
n-permutation? Answer the same question for reverse alternating 
permutations. 

(b) Let d,(n) be the number of alternating n-permutations with ex- 
actly k fixed points, and let dj(n) denote the number of reverse 
alternating n-permutations with exactly k fixed points. Prove that 
ifn > 4, then 

dtnja (2) = Dinj2)> 
and if n > 5, then 


Fina) /2)() = Di(n—1)/2)- 


(c) Prove that the number of alternating permutations of length 2n 
having k excedances and n fixed points is equal to the number of 
derangements of length n having k excedances. 


(d) Prove that don = di(n), and dj(n) = dj} (n). 
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21. (a) Find a formula for the number of derangements of length n that are 
even permutations minus the number of derangements of length n 
that are odd permutations. 


(b) Let m > 2 be a fixed positive integer, and let O0< r<m-—1bea 
fixed integer. Let p(n,m,r) be the probability that the number of 
cycles of a randomly selected n-permutation is of the form mq +r. 
Prove that 

; 1 
lim p(n,m,r) = —. 
n—- oo m 
22. Let n be a fixed positive integer, and let G,,,-(z) be defined as in Problem 
Plus 10, that is, as the generating polynomial of all r-derangements of 
length n with respect to their number of cycles. 


(a) Prove that for all negative integers m, and for all real numbers 
€ > 0, there exists a threshold N so that if n > N, then the 
polynomial G,,,(z) has a root in the interval (m— e€,m +). 


(b) Deduce a statement on r-derangements that is analogous with the 
claim of part (b) of Problem Plus 22. 


23. Recall that if S C [nm — 1], then 6(S) denotes the number of all permu- 
tations of length n whose descent set is equal to S. We have computed 
B(S) in Theorem 1.4. Let 8°%°(S) be the number of permutations of 
length n that have descent set 8 and consist of a single cycle of length 
n. 


Prove that for each n, there exists a collection F,, of subsets of [n — 1] 
so that lint, is4(\F,|/2"-*) = 1, and 


npeue(s!) 
B(S) 


In other words, for most subsets S, the ratio of the number of cyclic 
permutations with descent set S to the number of all permutations with 
descent set is close to 1/n. Note that if we drop the requirement that 
these permutations have descent set S, then the corresponding ratio is 
exactly 1/n. So, that ratio is fairly independent of the required descent 
set. 


lim max 
no SEF, 


-1)=0. 


DT 


Solutions to Problems Plus 


1. This result was first proved by Jeffrey Remmel in [275] as a corollary 
to a more general argument involving g-analogues. Other proofs can be 
found in [141] and (in a slightly different context), in [328]. 
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2. (a) The proofs are different for the cases of odd n and even n. The case 
of even n can be proved very similarly to Exercise 61. For odd n, 
however, we cannot solely rely on our previous methods, because 
2n + 2 is divisible by 4, and so there are permutations counted 
by SQan+2 whose odd part is empty. (And therefore, their even 
part is of length 2n + 2, and as such, cannot be found in shorter 
permutations). One can show, however, that the number of this 
kind of permutations is very small ifn > 8. 


(b) Equality holds if and only ifm = 1. As part [(a)] shows that 
SQeanti < SQan+2 for n > 8, this can be seen by checking the 
cases of odd n for n < 7. 


3. A more general version of this result is proved in [53]. 


4. The first combinatorial proof was given in [43]. A slightly simpler proof 
appears in [34]. 


5. (a) Formula (3.3) is clearly equivalent to 


Sl e(n + 1k +1)z* = (2 +1)(z+2)---(z +n) 
k=0 


n 


=nl(i+z)(1+5)--- (1+). 


Then c(n + 1,3) is the coefficient of z? on the right-hand side. The 
proof then follows by routine computations. 


(b) Similar to part (a). 
6. This result is proved in [53]. 


7. This can be proved like Problem Plus 5, with more complex computa- 
tions involving symmetric functions. See [123] or [109] for those details. 
Here C is the Euler constant, that is, 


8. We have seen in Lemma 1.17 that k!S(n,k) = 5-1 — i)”. 
We claim that all summands of the right-hand side are negligibly small 
compared to the last one. That is, we claim that for fixed & and fixed 
i € |[k], we have that 


(k — i)"(*) 


> 0 


as n — oo. This is clearly true as ie < 2*, and (k—i)/k <1. 
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9. This line of research was essentially initiated by Paul Edelman [150]. 


(a) We have b(n, k) =n —k. On one hand, b(n, k) < n—k& is easy to 
see for instance by induction. On the other hand, the permutation 
(12:--n—k+1)(n—k + 2)(n—k+3)---(n) shows that this is 
indeed possible. 


(b) We have 
(2) — [$1 +4, ifk < 131, 


(2) — (%"), if > 131. 

(c) Let us call a cycle (c1c2--+Cm) unimodal if the sequence of entries 
C1, €2,°**,Cm is unimodal. Let us call a permutation p unimodal if 
all its cycles (when started with their smallest elements) are uni- 
modal, and when the underlying sets of all cycles of p consist of 
intervals of positive integers. For instance, (1342)(58976) is a uni- 
modal permutation. Then it can be proved that b(n, k) is attained 
by p if and only if p is a unimodal permutation. This implies that 
for 1 <k <n-—1, the number m(n,k) we are looking for is given 


. * (n—k-1\ (k 
m(n, k) | A Ve 


r=0 


B(n,k) = 


(d) Ifn-1>k>[¥], then B(n,k) is attained by only one permuta- 
tion. This permutation p is given by 


n+1-4, ifie fl,n—-—k]U[K+1,n], 
pli) = 
i, fie [n—k+1,h] 
In other words, p is an involution that leaves the middle of the set 
[n] element-wise fixed. 
If k < | $], then the numbers M(n, k) we are looking for are given 


b 
y ten 3 a. ‘ @iunece 


r=0 
See [150] for further details, proofs and generalizations. 


10. This result is due to Francesco Brenti, and can be found in [96]. 


11. (a) This result (and the rest of the results in this exercise) is due to 
Francesco Brenti, and can be found in [97]. In fact, it is proved in 
that paper that 
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12. 


13. 


14. 


where d = (dj, do,--- ,d,) is the partition defining the conjugacy 
class D, the A;(z) are the Eulerian polynomials, and finally, zp = 
Ts, 2" m;(d)!. In this last formula, m;(d) is the multiplicity of 
i as a part of d. 

(b) It is proved in [97] that Ep(z) is symmetric. Therefore, it has 
to take its maximal value in the middle. This is then proved to 
be at (n — m)/2, where m is the number of fixed points in the 
partition defining D. (If this is not an integer, then there are 
two maximal values, and they are bracketing (n — m)/2. In other 
words, (n — m)/2 is always the center of symmetry of the sequence 
of coefficients of Ep(z).) One main tool of the proof is the result 
of Exercise 24 of Chapter 1. 


(c) Let D be a conjugacy class of S,,, and let the partition defining D 
be (di, d2,--- ,dx), where d; > r for i € [k]. Then it follows from 
part [(b)] that Ep(z) is a symmetric and unimodal polynomial 
with center of symmetry n/2 as permutations in D have no fixed 
points. That is, the center of symmetry does not depend on D. 
Our definitions imply 


Ep, k,rn(Z) = S- Ep(z), 
D 


where D ranges over all conjugacy classes that are given by par- 
titions with all parts larger than r. The claim then follows as the 
right-hand side is the sum of symmetric and unimodal polynomials 
with center of symmetry n/2. 
(d) As we have 
Ep,.(n) (2) = S- Ep,k,r,n(2); 
k>1 


part [(c)] implies our claim. 


The first proof of this result was given in [143] by the use of symmetric 
functions. Two further, and more bijective, proofs were given in [144], 
one of which is the proof of a special case of a more general theorem. 


This result is due to Richard Stanley and Ira Gessel [194]. Let Q;, be 
the set of Stirling permutations, that is, the set of all permutations 
pip2*+*Pon of the multiset {1,1,2,2,---,k,k} in which u < v < w and 
Pu = Pw imply py > py. Then the number B;,; is equal to the number 
of permutations p,p2-::Pan € Qz that have exactly 7 descents in the 
following sense: p; > pj;41 or j = 2k for exactly i values of 7 € [2k]. The 
above reference contains two proofs of this fact, a combinatorial one and 
an inductive one. 


A proof using symmetric functions can be found in [193]. It would still 
be interesting to find a bijective proof. 
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The first proof is due to Volker Strehl [305] who used the classic 
Robinson-Schensted bijection in his proof. We will discuss this bijection 
in Chapter 7. A very short proof using quasi-symmetric functions was 
given in [193]. 


This follows from the following general result of Ed Bender and Rodney 
Canfield [33]. Let 21, Z2,--- be a log-concave sequence of non-negative 
real numbers, and define the sequences A,, and P,, by 


5 Ane” — +) =m gh = exp 3 aie 


n>0 n>0 joi 7 


Then the A, are log-concave and the P,, are log-convex. 


In order to get our result from this theorem, set 7; = 1 for 7 < k, 


and set Z; = 0 otherwise. Then the far right-side of the above formula 


. : k-1 Zj24 eee . : 
simplifies to exp ( 5> soa 97 ht which is precisely the exponential gen- 


erating function for the numbers p,,,. In other words, in this special 
case, we get Py = pnk- 


This result is proved in [143] in two different ways. That paper also con- 
tains further generalizations, such as refinements of the proved equality 
with respect to descent sets. 


This result is due to Dominique Foata and Guo-Niu Han [175]. 


(a) We claim that p cannot have more than n—|.J| fixed points. Indeed, 
if 2 € D(p), then it is impossible for both i and i+ 1 to be fixed 
points. On the other hand, n—|J| fixed points are possible as shown 
by the increasing permutation, or the decreasing permutation if n 
is odd. 


(b) This result is due to Guo-Niu Han and Guoce Xin [212]. They first 
rewrite the statement by using the result of Problem Plus 18 to 
replace the number of descents by dez(p). After that, they give a 
one-paragraph proof for the new statement. This is remarkable, be- 
cause several of the upcoming Problems Plus will be consequences 
of this statement. 


— 
~ 
So 


If p is alternating, then it is impossible for both 27 — 1 and 2% to 
be fixed points of p at the same time. So p has at most [n/2] fixed 
points. This maximum is reached, for instance by permutations 
623154 and 6235417. If p is reverse alternating, then it is impossible 
for both 27 and 21+ 1 to be a fixed point of p. This means that p 
cannot have more than |n/2| +1 fixed points. This maximum is 
achieved by permutations like 15342 and 153426. 


(b) This result is due to Robin Chapman and Lauren Williams [113]. 
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22. 


(c) This result is due to Guo-Niu Han and Guoce Xin [212]. Let p 


wa 


be an alternating (2n)-permutation with n fixed points. It is then 
not difficult to prove that for each i € [n], either 22 — 1 is a fixed 
point of p, and po; < 2i, or 27 is a fixed point of p, and 2i — 1 is 
an excedance of p. Conversely, if every i € [n] has this property, 
then p is alternating. Now remove all fixed points from p, to get 
a derangement of an n-element subset of [2n]. If we replace the 
rth smallest entry of this derangement by r, we get a derangement 
@(p) of [n] that has the same number of excedances as p. It can 
then be proved that ¢ is bijective. 


The first proofs of these facts were given by Richard Stanley in 
[298], who asked for combinatorial proofs. Such proofs were then 
given in [212]. 


A short combinatorial proof is given by Robin Chapman [112]. 


Let us first assume that r = 0. Let a@ be an mth root of unity 
with the smallest non-zero argument, and set a = z in (3.3). Then 
sum the obtained formula for all mth roots of unity, that is, for 
all a’, with t = 0,1,---m—1. Using the summation formula of a 
geometric progression, we get that 


m-1 0 if a* £1, that is, if m is a divisor of k, 
t=0 m if a*® = 1, that is, if m is not a divisor of k. 


Therefore on the right-hand side, all summands c(n,k)(a‘)* will 
cancel except for those in which k& is a multiple of m; if k is a 
multiple of m, then the summands containing c(n, k) will add up 
to m-c(n,k). On the left-hand side, we will have a real number 
that is the sum of m terms, m — 1 of which have a negligibly 
small absolute value compared to the one in which z = 1, and has 
absolute value n!. Dividing both sides by n! and taking limits, our 
claim is proved. 

If r # 0, then the proof is very similar, but instead of simply 
adding the m equations obtained by setting z = a? in (3.3), we 
multiply each such equation by a~", and then sum them over 
t =0,1,---,m-—1. The only change is that on the left-hand side, 
all summands c(n,k)(a‘)* will cancel, except for those in which k 
is congruent to r modulo m. 


See On a balanced property of derangements [59]. The key idea is 
that the polynomial G,,,(z)/n! (as defined in Problem Plus 10) 
takes very small values for z = —1,—2,--- ,—r if r is small com- 
pared to n. Hence this polynomial must have a root close to such 
negative integers. 
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(b) The probability that the number & of cycles of a randomly selected 
r-derangement of length n is congruent to r modulo m converges 
to 1/m. The proof is analogous to that of part (b) of Problem Plus 
21. 


23. This result can be found in Section 5.2 of [162]. 


A 


In Any Way but This. Pattern Avoidance. 
The Basics. 


4.1 Notion of Pattern Avoidance 


In earlier chapters, we have studied inversions of permutations. These were 
pairs of elements that could be anywhere in the permutation, but always 
related to each other the same way, that is, the one on the left was always 
larger. 

There is a far-fetching generalization of this notion from pairs of entries to 
k-tuples of entries. Consider a “long” permutation, such as p = 25641387, 
and a shorter one, say g = 132. We then say that the 3-tuple of entries (2,6,4) 
in p forms a pattern or subsequence of type 132 because the entries (2,6,4) of 
p relate to each other as the entries 132 of q do. That is, the first one is the 
smallest, the middle one is the largest, and the last one is of medium size. 
The reader is invited to find a pattern of type 321 in p. On the other hand, 
there is no pattern of type 12345 in p; therefore, we will say that p avoids 
12345. 

The notion of pattern avoidance is at the center of this entire chapter, so 
we will make it formal. 


DEFINITION 4.1 Let qg = (m,4,°°:;d&) € Sz be a permutation, and 
letk <n. We say that the permutation p = (pi, p2,°-* ,Pn) € Sn contains q as 
a pattern if there are k entries pi,,Pir,°** , Pi, np so that ty < tg <+++ <i, 
and pi, < pi, of and only if da < q@. If p does not contain q, then we say that 
p avoids q. 


Example 4.2 
The permutation p = 57821346 contains the pattern g = 132 as shown in 
Figure 4.1. 


In other words, p contains q as a pattern if p has a subsequence of elements 
that “relate” to one another the same way the elements of q do. 
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FIGURE 4.1 
Containing the pattern 132. 


4.1.1 Permutation classes 


Let P be the infinite partially ordered set of all finite permutations ordered 
by pattern containment. That is, in this poset we have p <p q if gq contains 
pasa pattern. This P is an interesting poset and we will learn more about 
it in Section 7.2. Recall that a subset C of a partially ordered set R is called 
an ideal if for all elements x € C' and all elements y € R satisfying y < x, the 
inclusion y € C’ also holds. In other words, ideals are ” closed downward” . 


DEFINITION 4.3 If C is an ideal of the poset of P of all finite per- 
mutations ordered by pattern containment, then we will call C a permutation 
class. 


The set of all permutations that avoid a given pattern, the set of permu- 
tations that avoid a given set of patterns, or the set of all permutations that 
are shorter than a given bound are all examples of permutation classes. If the 
permutation class C consists of all permutations that avoid one given pattern 
q, then we call C = C(q) a principal permutation class. 


a 


4.2 Patterns of Length Three 


We have enumerated n-permutations with a given number of inversions and 
obtained rather precise results. The corresponding question, that is, a formula 
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for the number of n-permutations with a given number of occurrences of a 
pattern q, is in general too difficult. We will therefore first concentrate on the 
special case when this given number is zero. That is, we will try to find the 
number Av,,(qg) of n-permutations that avoid the pattern q. 

Obviously, we have Av,,(12) = Av,(21) = 1, so the first nontrivial case is 
that of patterns of length three. There are six such patterns, but as we will 
shortly see, there are many symmetries between them. 

Recall that for the permutation p = pip2---Pn, we define the reverse of 
p as the permutation p” = pnpn—1--:pi, and the complement of p as the 
permutation p° whose ith entry is n+ 1 — pj. 

It is clear that if a permutation avoids 123, then its reverse avoids 321, thus 
Av, (123) = Av,,(321). Similarly, if a permutation avoids 132, then its reverse 
avoids 231, its complement avoids 312, and the reverse of its complement 
avoids 213. Therefore, we also have Av,(132) = Av,(231) = Av,,(312) = 
Av, (213). 

If we can prove that Av,,(123) = Av,(132), then we will have the remark- 
able result that all patterns of length three are avoided by the same number 
of n-permutations. 


LEMMA 4.4 
For all positive integers n, the equality Av,(123) = Av,,(132) holds. 


PROOF ~ The are several ways to prove this first nontrivial result of the 
subject. The one we present here is due to Rodica Simion and Frank Schmidt 
[284]. Recall that an entry of a permutation that is smaller than all the entries 
that precede it is called a left-to-right minimum. Note that the left-to-right 
minima form a decreasing subsequence. 

We are going to construct a bijection f from the set of all 132-avoiding 
n-permutations to the set of all 123-avoiding n-permutations that leaves all 
left-to-right minima fixed. 

The map f is defined as follows: Keep the left-to-right minima of p fixed, 
and write all the other entries in decreasing order in the positions between the 
left-to-right minima. The obtained permutation f(p) is always 123-avoiding 
as it is the union of two decreasing subsequences, one of which is the sequence 
of all left-to-right minima, and the other is the decreasing sequence into which 
we arranged the remaining entries. 


Example 4.5 
If p = 67341258, then the left-to-right minima of p are 6, 3, and 1; therefore, 
f(p) = 68371542. J 


We would like to point out that the left-to-right minima of p and f(p) are 
the same, even if some other entries of p have moved. Indeed, we can say that 
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f simply rearranges the m entries that are not left-to-right minima pair by 
pair. That is, whenever we (impersonating the function f) see a pair of these 
entries that are not in decreasing order, we swap them. This algorithm stops 
in at most (3) steps. Moreover, each step of this algorithm moves a smaller 
entry to the right and a larger one to the left and therefore never creates a 
new left-to-right minimum. 

We note that this is the only 123-avoiding permutation with the given set 
and position of left-to-right minima. Indeed, if there were two entries x and y 
that are not left-to-right minima and form a 12-pattern, then the left-to-right 
minimum z that is closest to x on the left, and the entries x and y, would 
form an increasing sequence. 

Now we prove that f is a bijection by showing that it has an inverse. Let q 
be an n-permutation that avoids 123. Keep the left-to-right minima of q fixed, 
and fill in the remaining positions with the remaining entries, moving left-to- 
right, as follows. At each step, place the smallest element not yet placed that 
is larger than the closest left-to-right minima on the left of the given position. 
Call the obtained permutation g(q). 


Example 4.6 

If q = 68371542, then the left-to-right minima of q are 6, 3, and 1. To the 
empty slot between 6 and 3, we put the smallest of the two entries that are 
larger than 6, that is, 7. In the empty slot between 3 and 1, we put the 
smallest entry not used yet that is larger than 3, that is, 4. Immediately on 
the right of 1, we place the smallest entry not used yet that is larger than 1, 
that is, 2. We finish this way, by placing 5 and 8 in the remaining slots, to 
get g(q) = 67341258. 


The obtained permutation is always 132-avoiding. Indeed, if there were 
a 132-pattern in it, then there would be one that starts with a left-to-right 
minimum, but that is impossible as entries larger than any given left-to-right 
minimum are written in increasing order. 

Note again that g(q) is the only 132-avoiding permutation that has the 
same set and position of left-to-right minima as qg. Indeed, if at any given 
instance, two entries u < v that are larger than a left-to-right minimum a 
were in decreasing order, then avu would be a 132-pattern. 

This proves that g(f(p)) = p, implying that f is a bijection, and proving 
our theorem. 


The techniques and notions used in the above proof will be used so often 
in the coming sections that it is worth visualizing them. Figure 4.2 shows 
our running example, the permutation p = 67341258, and its image, f(p) = 
68371542. 

Note again that the left-to-right minima (the decreasing sequence of gray 
circles) are left fixed. In f(p), all the remaining entries form a decreasing 
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A 132-avoiding permutation and its image. 


sequence, while in p, all entries larger than a given left-to-right minimum 
form an increasing sequence. 

Now that we know that Av,(q) does not depend on qg as long as the length 
of q is three, it suffices to determine Av,,(q) for one possible choice of q of 
length three. 


THEOREM 4.7 
For all positive integers n, the equality 


2n 
Avn(132) = Ch = (n) 


holds. 


PROOF Set C, = Av,,(132). Suppose we have a 132-avoiding n- 
permutation in which the entry n is in the ith position. Then any entry 
to the left of nm must be larger than any entry to the right of n. Indeed, if x 
and y violate this condition, then xny is a 132-pattern. Therefore, the set of 
entries on the left of n must be {n —i+1,n—i4+2,---,n—1}, and the set of 
entries on the right of n must be [n — 7]. Moreover, there are Cj_ possibilities 
for the order of entries to the left of n and C,_; possibilities for the order of 
entries on the right of nm. Summing over all 7 we get the recurrence relation 


n-1 
CpG 5C,.5. (4.1) 
1=0 


Therefore, if C(z) = ae C,z" is the ordinary generating function of the 
numbers C,,, then (4.1) implies C?(z)z +1 = C(z). That functional equation 


in turn yields 
1—-V1—4z 


a 22 


(4.2) 
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By standard methods this yields 


aes 3 x(n?) oo > n : 1 (*") - Soe 


n=1 n=0 


and the statement of the theorem is proved. | 


Because of the trivial identities Av,(132) = Av,(231) = Av,(312) = 
Av,(213), and Av,(123) = Av,,(321), we have proved the following. 


COROLLARY 4.8 
Let q be a pattern of length three. Then 


Avn(q) = Ch = 


The numbers C,, are called the Catalan numbers, and are perhaps the most 
famous counting sequence in combinatorics. Richard Stanley [297] has col- 
lected over 150 different classes of objects that are enumerated by these num- 
bers. We will shortly see that the condition that q is of length three is of 
crucial importance in several aspects. 

The following definition will make future discussion of pattern avoiding 
permutations simpler. 


DEFINITION 4.9 We say that the permutation patterns q and q' are 
Wilfequivalent if Avn(q) = Avn(q’) for all n. 


So Corollary 4.8 implies that all patterns of length three are Wilf-equivalent 
to each other. 

We end this section by a simple consequence of Lemma 4.4 that we will need 
later in this chapter. Recall that we have seen in the proof of Lemma 4.4 that 
a 132-avoiding permutation is completely determined by the set and position 
of its left-to-right minima. The following proposition makes this observation 
explicit. 


PROPOSITION 4.10 
Let p = pip2-::Pn be a permutation of length n that avoids 132. Let the 
ordered pairs of words (u(p),v(p)) of length n be defined as follows. The ith 
letter of u(p) is A if p; ts a left-to-right minimum in p, and B otherwise. The 
ith letter of u(p) is A if the entry of value i is a left-to-right minimum in p, 
and B otherwise. 

Then the map r(p) = (u(p), v(p)) és injective. 
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Example 4.11 
If p = 43512, then u(p) = AABAB and v(p) = ABAAB. OJ 


Taking reverse permutations, we get a dual version of Proposition 4.10 that 
we will use soon as well. Therefore, we announce it on its own. 


PROPOSITION 4.12 
Let p = pip2-::pn be a permutation of length n that avoids 218. Let the 
ordered pairs of words (x(p),y(p)) of length n be defined as follows. The ith 
letter of x(p) is C if p; is not a right-to-left maximum in p, and D otherwise. 
The ith letter of y(p) is C if the entry of value i is not a right-to-left maximum 
in p, and D otherwise. 

Then the map s(p) = (a(p), y(p)) és injective. 


Example 4.13 
If p = 35412, then x(p) =CDDCD, and y(p)=CDCDD. IT 


Dn 


4.3. Monotone Patterns 


After obtaining satisfying results for the case of patterns of length three, the 
reader may think that we will now turn to patterns of length four, and then 
to the general case, and obtain similarly strong results. Unfortunately, that is 
quite a mountainous task. As we will shortly see, finding an exact formula is 
difficult even for patterns of length four, and is an open problem for all longer 
patterns. However, a long-standing conjecture (proved in 2003 [250] by Adam 
Marcus and Gabor Tardos) connected all patterns by claiming that no matter 
what q is, the number of n-permutations avoiding q is very small compared 
to nl. 


CONJECTURE 4.14 [Stanley—Wilf conjecture, 1980] Let q be any 
pattern. Then there exists a constant cq so that for all positive integers n, 


Avn(q) Se (4.4) 


nm 
a 
It is worth pointing out that Vaughan Pratt [267] almost made this conjec- 
ture explicit in 1973! 
Note that the conjectured number cj is very small compared to the number 
of all n-permutations, which is n!. In other words, this is a quite ambitious 
conjecture. 
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The following conjecture may look even more ambitious, but that is a false 
appearance. 


CONJECTURE 4.15 [Stanley—Wilf conjecture, alternative version] Let 
q be any pattern. Then the limit 
L(q) = lim VAvn(q) 


n—->co 


exists. 


The number L(q) is a crucial piece of information about the pattern gq, and 
the principal permutation class generated by it. We will usually call it the 
growth rate of the class of g-avoiding permutations, though in the past it was 
sometimes called the Stanley-Wilf limit of qg. The advantage of the termi- 
nology growth rate is the following. As we will see, for principal permutation 
classes, the limit limp... *~/Avn(q) always exists. However, for permutation 
classes in general, it may not. This warrants the following definition. 


DEFINITION 4.16 IfC is a permutation class, and |C,,| is the number 
of elements of C that are of length n, then 


lim sup V/|C,,| 
is called the growth rate of C. 


The first published proof of the fact that the above two versions of the 
conjecture are equivalent is given in [17]. On one hand, it is obvious that 
Conjecture 4.15 implies Conjecture 4.14. On the other hand, to prove the 
implication in the other direction, we claim that Avin(q)Avn(q) < AVm4n(q), 
for all patterns q, and all positive integers m and n. 


DEFINITION 4.17 Let p be a permutation of length m, and let p’ be a 
permutation of length n. Then their sum is the permutation p © p’ of length 
m+n given by 


(pep')i= 
M+Ppi_m, fmt+1<icm+n. 


Example 4.18 
If p = 213 and p' = 3142, then p@ p’ = 2136475. J 


Sometimes, the dual concept of skew sum is useful. 
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DEFINITION 4.19 Let p be a permutation of length m, and let p' be 
a permutation of length n. Then their skew sum is the permutation p © p’ of 
length m+n given by 


(Pep) = 
Diem, fm+1<icmen. 


Example 4.20 
If p = 213 and p’ = 3142, then pO p’ = 6573142. 


Let us assume, without loss of generality, that the first entry of q is larger 
than the last entry of g. Let p and p’ be permutations of length m and n, 
respectively, that avoid q. 

Then the permutation p @ p’ € Avm4n clearly avoids q, proving (injec- 
tively) that Avm(q)Avn(q) < AVnim(q). Therefore the sequence ~/Av»,(q) is 
bounded and monotone, and must thus be convergent. 

As we have mentioned, the Stanley—Wilf conjecture was proved to be true 
[250] in 2003. We will present that spectacular proof in Section 4.5. For now, 
we will consider some special cases in which more precise results are available. 

We have seen in the previous section that the Stanley—Wilf conjecture is 
true if q is of length three, with c, = 4. Indeed, C, < (*") < 4". There 
is only one other class of patterns for which it is similarly easy to prove the 
Stanley—Wilf conjecture, namely monotone patterns. 


THEOREM 4.21 
For all positive integers n and k > 2, we have 


Avn(123---k) < (k —1)?”. 


PROOF Let us say that an entry x of a permutation is of rank 7 if it 
is the end of an increasing subsequence of length 7, but there is no increasing 
subsequence of length i+ 1 that ends in x. Then for all i, elements of rank 
i must form a decreasing subsequence. Therefore, a g-avoiding permutation 
can be decomposed into the union of k — 1 decreasing subsequences. 

This means that a permutation p avoiding 12---k is completely determined 
by the following two ordered partitions of [n] into k — 1 blocks. (An ordered 
partition is a partition whose set of blocks is linearly ordered.) 


1. The partition P, of the set of n entries of p, into the blocks of entries 
of rank 1, the block of entries of rank 2, and so on. 
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2. The partition P: of the n positions of p, into the block of positions 
containing entries of rank 1, the block of positions containing entries of 
rank 2, and so on. 


Indeed, once these partitions are known, the entries of rank i must occupy 
the positions assigned to them, in decreasing order. 

There are at most (k — 1)” possibilities for each of P,; and P2, proving our 
theorem. 


Note that for k = 3, we get that Av,,(123) < 4”, agreeing with our earlier 
results. As we have seen that Av,(123) = C,,, we see that the constant 4 
provided by Theorem 4.21 is actually the best possible constant. This is not 
an accident. 

Indeed, Theorem 4.21 has a stronger version, which needs heavy analytic 
machinery, and therefore will not be proved here. The result is significant as 
it shows that no matter what k is, the constant (k — 1)? cannot be replaced 


by a smaller number. Recall that functions f(nm) and g(n) are said to be 
fn) _ 4 


asymptotically equal if limp. Cle 


THEOREM 4.22 
[272] For all k > 2, the asymptotic equality 


(k — 1)?" 


Av, (1234 ware k) = Ana aR 2 


holds. Here 


Ak = if / vo [Dler ta: ,@E) e~ (k/2)2*)2 dans dary stadt, 


UL Mw~> S Lk 
where D(a1,22,°++ ,k) = [[jcj(ai — 25), and ye = (1/V27)*? - Kk? /2. 


Note that the multiple integral in the above formula evaluates to a constant. 


4.4 Patterns of Length Four 


In this section we will study the number Av,,(q) for patterns of length four. 
There are 24 patterns of length four, but there are many (trivial and nontriv- 
ial) Wilf-equivalences between them. First, we will use these equivalences to 
significantly decrease the number of patterns that need individual attention. 

By taking reverses, complements, and reverse complements, we can restrict 
our attention to those patterns of length four in which 
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(a) the first element is smaller than the last one and 
(b) the first element is 1 or 2. 


This still leaves us 10 patterns, 1234, 1243, 1324, 1342, 1423, 1432, 2134, 
2143, 2314, and 2413. Note that if p contains q, then the inverse of p clearly 
contains the inverse of q (as the inverse of a permutation matrix is its trans- 
pose), so Av,,(q) = Avn(q~*). Therefore, we can drop 1423, too, as its inverse 
1342 remains on the list. Similarly, we can drop 2314 as its complement is 
3241 and the reverse of that is again 1423. 

We cannot proceed further without new tools. The following general the- 
orem of Backelin, West, and Xin has a quite involved proof, but some of its 
special cases are easier. 


THEOREM 4.23 
[26] Let k be any positive integer, and let q be a permutation of length r. Then 
for all positive integers n, the equality 


Avn((12-+-k) @q) = Avn((k(k — 1)---1) @ q) 
holds. 


For instance, if r = 2, k = 2, and q = 34, then Theorem 4.23 says that 
Av, (1234) = Av,,(2134). If g = 43 and the other parameters do not change, 
then Theorem 4.23 says that Av,(1243) = Av,,(2143). As 2134 and 1243 are 
reverse complements of each other, the patterns 2134, 2143, and 2134 can all 
be removed from our list. Now let k = 3, and r = 1. Then clearly q = 4, 
and Theorem 4.23 shows that Av,,(1234) = Av,,(3214). Therefore, by taking 
complements, we can remove the complement of the reverse of 3214, that is, 
the pattern 1432, from our list. 

This leaves us with the patterns 1234, 1324, 1342, and 2413. The following 
result of Zvezdelina Stankova [288] eliminates one more pattern. 


LEMMA 4.24 
For all positive integers n, the equality 


Avy (1342) = Av, (2413) 


holds. 


We note that Stankova proved her result in the equivalent form of Av,,(4132) = 
Av, (3142). 

So we are left with three patterns, 1234, 1342, and 1324. It is high time 
that we took a look at numerical evidence computed by Julian West [325]. 
This evidence shows that as n grows starting at n = 1, 
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e for Av,,(1342) we have 1, 2, 6, 23, 103, 512, 2740, 15485 
e for Av,(1234) we have 1, 2, 6, 23, 103, 513, 2761, 15767 
e for Av,,(1324) we have 1, 2, 6, 23, 103, 513, 2762, 15793. 


These data are startling for at least two reasons. First, the numbers Av,,(q) 
are no longer independent of g. That is, there are some patterns of length four 
that are easier to avoid than others. Second, the monotone pattern 1234 does 
not provide either extremity. That is, it seems that the monotone pattern is 
neither the easiest nor the hardest one to avoid. 

The numerical evidence shown above also raises some intriguing questions. 


QUESTION 4.25 Is it true that for alln > 7, the inequality 
Av, (1234) < Av,(1324) 
holds? 


QUESTION 4.26 Is it true that for all n > 6, the inequality 
Av, (1342) < Av,,(1234) 
holds? 


QUESTION 4.27 In general, what makes a pattern easier to avoid than 
another pattern? 


QUESTION 4.28 Is it true that if Avn(qi) < Avn(q2) for some n, then 
for all N > n, we have Avn(qi) < Avn(q2)? 


In this section, we will answer Questions 4.25 and 4.26 in the affirmative. 
There is no known answer for Question 4.27, but we will mention some related 
interesting facts. The answer for Question 4.28 is, in general, negative. The 
first counterexample [289] is for two patterns q; and qe of size five, and for 
n = 12. So whether a pattern is easy or hard to avoid depends in some cases 
not just on the pattern itself, but also on n. 


4.4.1 Pattern 1324 


The following result is the earliest example [44] in which a pattern was shown 
to be more restrictive than another pattern of the same size. 


THEOREM 4.29 
For alln > 7, we have 


Avy (1234) < Avn (1324). 
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The first step in our proof is the following classification of all n-permutations. 


DEFINITION 4.30 We will say that two permutations x and y have 
the same profile if 


e the left-to-right minima of x are the same as those of y, and 


e the left-to-right minima of x are in the same positions as the left-to-right 
minima of y, and 


e the above two conditions hold for the right-to-left maxima of x and y as 
well. 


Note that profiles are equivalence classes. 


Example 4.31 

Permutations x = 51234 and y = 51324, have the same profile, but z = 24315 
and v = 24135 do not, as the third entry of z is not a left-to-right minimum, 
whereas that of v is. 


The outline of our proof is going to be as follows: we show that for each 
(nonempty) profile T’, there is exactly one 1234-avoiding permutation and at 
least one 1324-avoiding permutation having profile T. Then we show that “at 
least one” means “more than one” at least once, completing the proof. 

The first half of our argument is simple. 


LEMMA 4.32 
For each non-empty profile T, there is exactly one 1234-avoiding permutation 
having profile T. 


PROOF Let us assume that we have chosen a profile 7’, that is, we 
fixed the positions and values of all the left-to-right minima and right-to-left 
maxima. It is clear that if we put all the remaining entries into the remaining 
slots in decreasing order, then we get a 1234-avoiding permutation. Indeed, 
the permutation obtained this way consists of three decreasing subsequences, 
that is, the left-to-right minima, the right-to-left maxima, and the remaining 
entries. If the permutation just constructed contained a 1234-pattern, then by 
the pigeon-hole principle two of the entries of that 1234-pattern would be in 
the same decreasing subsequence, which would be a contradiction. Note that 
if T is non-empty, then we can indeed write the remaining entries in decreasing 
order without conflicting with the existing constraints — otherwise T’ would 
be empty. This can be seen as in the proof of Lemma 4.4. Therefore, at least 
one 1234-avoiding permutation has profile 7’. 
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On the other hand, we claim that the decreasing order of the remaining 
entries is the only one that will result in a 1234-avoiding permutation. Indeed, 
if we put two of the remaining entries, say a and 0, in increasing order, then 
together with the rightmost left-to-right minimum on the left of a@ and the 
leftmost right-to-left maximum on the right of b they would form a 1234- 
pattern. 


The second half of our argument is somewhat subtler. 


LEMMA 4.33 
Each nonempty profile contains at least one 1824-avoiding permutation. 


PROOF | First note that if a permutation contains a 1324-pattern, then we 
can choose such a pattern so that its first element is a left-to-right minimum 
and its last element is a right-to-left maximum. Indeed, we can just take any 
existing pattern and replace its first (last) element by its closest left (right) 
neighbor that is a left-to-right minimum (right-to-left maximum). Therefore, 
to show that a permutation avoids 1324, it is sufficient to show that it does 
not contain a 1324-pattern having a left-to-right minimum for its first element 
and a right-to-left maximum for its last element. Such a pattern will be called 
a bad pattern. Also note that a left-to-right minimum (right-to-left maximum) 
can only be the first (last) element of a 1324-pattern. 

Now take any 1324-containing permutation. By the above argument, it has 
a bad pattern. Interchange the second and third entries of this bad pattern. 
Observe that we can do this without violating the existing constraints, that 
is, no element x goes on the left of a left-to-right minimum that is larger 
than x, and no element y goes on the right of a right-to-left maximum that 
is smaller than y. The resulting permutation has the same profile as the 
original one because the left-to-right minima and right-to-left maxima have 
not been changed. Repeat this procedure as many times as possible, that 
is, as long as 1324-patterns can be found. Note that crucially, each step of 
the procedure decreases the number of inversions of our permutation by at 
least one. Therefore, we will have to stop after at most (S steps. Then, the 
resulting permutation will have the same profile as the original one, but it 
will have no bad pattern and therefore no 1324-pattern, as we claimed. 


We are only one step away from proving Theorem 4.29. We have to show 
that ifn > 7, then classes that contain more than one 1324-avoiding permuta- 
tions indeed exist. Let n = 7, then the class 3*1*7*5 (with the stars denoting 
the positions of the remaining entries) contains two such permutations, namely 
3612745 and 3416725. If n > 7, then put the entries n,n —1,--- ,8 in front 
of 3*1%*7*5, with no additional stars. The obtained class will again contain 
two 1324-avoiding permutations, coming from the subwords 624 and 462 of 
the remaining entries. This completes the proof of Theorem 4.29. 
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At this point, readers with a penchant for asymptotic results will surely 
say something like “OK, Av,,(1324) is larger than Av,,(1234), but is it really 
significantly larger?”. That is, could it be that Av,(1234) and Av,,(1324) 
are in fact asymptotically equal, meaning that (Av, (1234) /Av,(1324)) > 1? 
The following proposition answers this question in the negative. 


PROPOSITION 4.34 
The sequences Av,,(1234) and Av,,(1324) are not asymptotically equal. 


PROOF See the solution of Exercise 29. | 


What can we say about £(1324)? In [5], it is proved that £(1324) > 9.47. 
In [38], David Bevan improved that lower bound by proving the inequality 
£(1324) > 9.81. Note that it follows from Theorem 4.22 that L(1234) = 9. 
Therefore, the sequences Av,,(1234) and Av,,(1324) have different growth 
rates, and the sequence Av,,(1234)/Av,,(1324) converges to 0 at an expo- 
nential rate. 


4.4.1.1 Exponential Upper Bound 


In the previous subsection, we have seen that it is easier to avoid 1324 than 
1234, but how much easier? We will now discuss three results concerning 
upper bounds for £(1324), in the order they were discovered. 
We start by presenting a result of Anders Claesson, Vit Jelinek, and Einar 
Steingrimsson [119] showing that Av,,(1324) < 16” for all positive integers n. 
The following simple definition will be crucial for the proof. 


DEFINITION 4.35 Let p’ be a permutation of length k and let p’ be 
a permutation of length m, withk+mz=n. We say that the n-permutation 
Dp = Pip2-:*Pn is a merge of p’ and p” if there is a partition of the set [n] 
into two blocks, I = {i1,%2,---,in} and J = {ji,jo,-++ jm}, so that the 
subsequence pi, Pip +++ Di, 18 a p'-pattern, and the subsequence Pj, Pjo°**DPjm 1S 
a p"'-pattern. 


In other words, p is a merge of p’ and p” if we can color each entry of p red 
or blue so that the red entries form a p’-pattern, and the blue entries form a 
p’-pattern. 


Example 4.36 

The permutation p = 4751326 is a merge of p’ = 123 and p” = 4132, as can be 
seen by choosing J = {1,3,7} and J = {2,4,5, 6}, leading to the subsequences 
456 and 7132. 
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The following lemma proves an upper bound for the number of permutations 
avoiding a pattern that is a merge of two shorter patterns. An earlier, less 
general version of this lemma was first used in [60]. The version we present is 
slightly less general than that of [119], but will suffice for our purposes. 


LEMMA 4.37 
Let q, q, and q" be three permutation patterns so that every q-avoiding per- 
mutation can be obtained as a merge of a q'-avoiding permutation and a q"’- 
avoiding permutation. 

Let a and b be positive constants so that for all positive integers n, the 
inequalities Avn(q’) < a” and Avn(q’) < 6” hold. Then for all positive 
integers n, the inequality 


Avn(q) < (Va+ vb) _ 


holds. 


PROOF Let p be a q-avoiding permutation of length n. Then, we can 
color each entry of p red or blue so that the subsequence of all red entries avoids 
qi and the subsequence of all blue entries avoids qz. If there are exactly k red 
entries, then this leaves less than (?)7akpn-* possibilities for p. Indeed, there 
are (7) choices for the set of red entries, and there are (j) choices for the set 
of positions of the red entries. 

Summing over all possible values of k, this shows that 


| 
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In order to be able to use Lemma 4.37 to prove an upper bound for Av,, (1324), 
we need to show that each 1324-avoiding permutation is a merge of two per- 
mutations avoiding shorter patterns. The following lemma, which is a special 
case of a lemma in [119], accomplishes that. 
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LEMMA 4.38 
Let p be a 1824-avoiding permutation. Then p is a merge of a 132-avoiding 
permutation and a 213-avoiding permutation. 


PROOF Let p = pip2-::Pn be a 1324-avoiding permutation. Let us 
color the entries of p one-by-one, from left to right, according to the following 
rules. 


1. If coloring p; red would create a 132-pattern with all red entries, then 
color p; blue, and 


2. if there already is a blue entry smaller than p;, then color p; blue; 
3. otherwise, color p; red. 


It is obvious from the first rule that the subsequence of red entries will 
indeed avoid 132. We claim that the subsequence of blue entries will avoid 
213. 

Let us assume that this is not the case, that is, there exist blue entries 
bz, 61, b3 (located in this order from left to right) that form a 213-pattern. 
Let 6; = p,. Then let p, be a blue entry such that s < t and p, < , and 
ps is minimal among entries satisfying these conditions. That means that p, 
was colored blue because of the first rule, that is, coloring it red would have 
completed a red 132-pattern consisting of the entries r; and r3, which are red, 
and p,. Note that in particular, ry < ps < py = 1. 

Now there are two possibilities. If r; is on the left of b2, then 71b2b,b3 forms 
a 1324-pattern in p, which is a contradiction. If r; is on the right of bo, then 
of course, rg is also on the right of bg. That implies that r3 < bg; otherwise, 
the second rule would force rz to be blue. In particular, rz < b3. This means 
that rir3p.b3 is a 1324-pattern in p, which is again a contradiction. 


We can now easily prove the result promised at the beginning of this sub- 
section. 


THEOREM 4.39 
[119] For all positive integers n, the inequality 


Av, (1324) < 16” 
holds. 


PROOF Use Lemma 4.37 with q, = 182, q2 = 218, and note that 
Av,(132) = Av,(213) = C,, < 4” as we proved in Corollary 4.8. 


Reading the proof of Theorem 4.39, it is tempting to think that there is 
a tremendous waste in our methods. In particular, the 2”-upper bound for 
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the set and position of the red entries, coming from Lemma 4.37, seems to be 
wasteful. The following result refines the techniques used in proving Theorem 
4.39, and significantly improves its exponential upper bound. 


THEOREM 4.40 
For all positive integers n, the inequality 


Avy (1324) < (7+ 4V3)” 


holds. 


Let us color each entry of the 1324-avoiding permutation p = pip2--- pn red 
or blue using the same rules as in the proof of Theorem 4.39. Furthermore, 
let us mark each entry of p with one of the letters A, B, C, or D as follows. 


1. Mark each red entry that is a left-to-right minimum in the partial per- 
mutation of red entries by A, 


2. Mark each red entry that is not a left-to-right minimum in the partial 
permutation of red entries by B, 


3. Mark each blue entry that is not a right-to-left maximum in the partial 
permutation of blue entries by C’, and 


4. Mark each blue entry that is a right-to-left maximum in the partial 
permutation of blue entries by D. 


Call entries marked by the letter X entries of type X. Let w(p) be the 
n-letter word over the alphabet {A, B,C, D} defined above. In other words, 
the ith letter of w(p) is the type of p; in p. Let z(p) be the n-letter word over 
the alphabet {A, B,C, D} whose ith letter is the type of the entry of value i 
in p. 


REMARK 4.41 Note that the function f(p) = (w(p), z(p)) in fact applies 
the map r of Proposition 4.10 to the string p;yeq of red entries of p, and the 
map s of Proposition 4.12 to the string ppiue of blue entries of p. So given 
f(p) = (w(p), z(p)), we can immediately recover r(preq) and 8(Pptue). Indeed, 
1’ (Drea) is the pair of subwords of w(p) and z(p) that consist of letters A and 
B, whereas s(ppiue) is the pair of subwords of w(p) and z(p) that consist of 
letters C and D. 

Conversely, if we are given r(preq) = (U(Prea),U(Pread)) and s(potue) = 
(x(Poiue)s ¥(Polue)), and we know in which positions of p the red entries are, 
and entries of which value of p are red, we can recover f(p) as follows. Shuffle 
the words u(prea) and x(ppiue) so that letters A and B are in positions that 
belong to red entries in p, and shuffle the words v(prea) and y(Ppiue) so that 
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letters A and B are in positions j for which the entry of value 7 is red in p. 


Example 4.42 

Let p = 3612745. Then the subsequence of red entries of p is 36127, the 
subsequence of blue entries of p is 45, so w(p) = ABABBCD, while z(p) = 
ABACDBB. 


The following lemma shows a property of w(p) that will enable us to improve 
the upper bound on Av,,(1324). Let us say that a word w has a CB-factor if 
somewhere in w, a letter C' is immediately followed by a letter B. 


LEMMA 4.43 
If p ts 1324-avoiding, then w(p) has no CB-factor. 


PROOF _ Let us assume that in p = pjp2--- pn, the entry p; is of type C, 
while the entry p,;+1 is of type B. That means that p; > pj, otherwise the 
fact that p; is blue would force p;,; to be blue. Furthermore, since p; is not a 
right-to-left maximum, there is an entry d on the right of p; (and on the right 
of pi41) so that p; < d. Similarly, since p;+1 is not a left-to-right minimum, 
there is an entry a on its left so that a < pj,1. However, then ap;p;j+id is a 
1324-pattern, which is a contradiction. 


LEMMA 4.44 
If p is 1324-avoiding, then there is no entry i in p so that i is of type C and 
i+1 is of type B. 


PROOF Analogous to the proof of Lemma 4.43. If such a pair existed, 7 
would have to be on the right of i+ 1, since i is blue and i+ 1 is red. As 7 is 
not a right-to-left maximum, there would be a larger entry d on its right. As 
1+1 is not a left-to-right minimum, there would be a smaller entry a on its 
left. However, then a(i + 1)id would be a 1324-pattern. 


LEMMA 4.45 
Let hy, be the number of words of length n that consist of letters A, B, C and 
D that have no CB-factors. Then we have 


H(z) =} hn2” = 35 : 


—Ag4+ 22° 
n>0 
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This implies 


he eae (2+ v3)" + ree (2- v3)". (4.5) 


PROOF This is straightforward, using the recurrence relation hnio = 
4hn+1 — hyn for n > 0, with ho = 1 and hy = 4. 


The following, simple, but crucial, lemma tells us that the ordered pair 
(w(p), z(p)) completely determines the 1324-avoiding permutation p. 


LEMMA 4.46 
Let Av,,(1324) be the set of all 1324-avoiding n-permutations. Then the map 
f : Aun (1324) — Hn x Hn, given by f(p) = (w(p), z(p)) is injective. 


PROOF _ Let (w,z) © Hn X Hn, and let us assume that f(p) = (w, z), 
that is, that w(p) = w, and z(p) = z for some p € Av,,(1324). 

Then w tells us for which indices 7 the entry p; will be of type A, namely 
for the indices i for which the ith letter of w is A. Similary, w tells us the 
indices j for which the entry p; is of type B, type C, or type D. 

After this, we can use z to figure out which entries of p are of type A, type 
B, type C or type D. 

Now let wap (resp. zag) be the subword of w (resp. z) that consists of 
all the letters A and B in w (resp. z). In other words, the pair (waz, z4B) 
contains all information about the red entries of p. It then follows from Propo- 
sition 4.10 that there exists at most one 132-avoiding permutation p’ for which 
r(p’) = (waB, 24B). 

Define wcp and zcp in an analogous manner. Then Proposition 4.12 shows 
that there exists at most one 213-avoiding permutation p” for which s(p”) = 
(wep, 2cD). 

It is now immediate from Remark 4.41 that f is injective. Indeed, if f(p) = 
(w,z), then the red entries of p must form the unique permutation p’ for 
which r(p’) = (wap,zap), and the blue entries of p must form the unique 
permutation p” for which s(p”) = (woep,zcp). Finally, as we said in the 
second and third paragraphs of this proof, the pair (w, z) uniquely determines 
the set and positions of red entries of p, and the set and positions of blue entries 
of p. 


We are now ready to prove Theorem 4.40. 
PROOF (of Theorem 4.40) The fact that Av,,(1324) < h? is immediate 


from the injective property of f that we have just proved in Lemma 4.46. In 
order to complete the proof of the first inequality, note that the image of f 
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consists of ordered pairs (w(p), z(p)) in which both w(p) and z(p) starts with 
an A, since both p; and 1 are always red, and left-to-right minima within 
the string of red entries (and even in all of p). The rest follows from formula 
(4.5), since in that formula, the second summand is negative, and in the first 
summand, the coefficient (3 + 2,/3)/6 is smaller than the base (2+ V3). | 


Theorem 4.40 shows that L(1324) = 7+4V3 < 13.93. Exploring some 
dependencies between the words w(p) and z(p), this upper bound can be 
improved to £(1324) < 13.73718. See [72] for details. 

An even sharper upper bound was found by David Bevan, Robert Brignall, 
Andrew Elvey Price and Jay Pantone [39], who proved that [(1324) < 13.5 
holds. The first step in their proof is the notion of a domino. 


DEFINITION 4.47 A domino of size n is a 1324-avoiding permutation 

p of length n and an integer k € [0,n] so that the entries of p that are larger 
than k form a 213-avoiding permutation (the top part of the domino), and 
the entries of p that are at most k (the bottom part of the domino) form a 
182-avoiding permutation. 


LEMMA 4.48 


The number of dominoes of size n is 


d= 2(3n + 3)! 
"(n+ 2)(2n +3) 

Example 4.49 

There are 22 dominoes of size three. Indeed, there are six permutations of 

length three, and they all avoid 1324. There are four choices for k, and almost 

all of them can be made with almost all permutations of length three. There 


are only two exceptions. When p = 213, then k cannot be 0, and when 
p = 182, then k cannot be three. 


See [39] for a proof of Lemma 4.48. 

We will now show how to decompose a 1324-avoiding permutation into a 
sequence of dominoes. Let Av, (q) denote the set of all permutations of length 
n that avoid the pattern g. Let p € Av,(1324). We will define a subsequence 
@1,Q2,°*: , a4, of p as follows. Let a, = py, the leftmost entry of p. 


e If 7 is even, then let a; be the largest entry of p that plays the role of 1 
in a 213-pattern consisting only of entries weakly on the right of aj_1. 


e Ifz > 1 is odd, then let a; be the leftmost entry of p that plays the role 
of 2 in a 132-pattern consisting only of entries weakly below a;_1. 
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FIGURE 4.3 
The domino decomposition of p = 3612745. 


The sequence of the a; ends when no entry satisfies the requirements to be 
the next a;. The domino decomposition of p is obtained by inserting horizontal 
lines immediately above a; for i even, and vertical lines immediately to the 
left of a; for 2 odd, if 2 > 1. See Figures 4.3 and 4.4 for illustrations. 

Note that it follows from the 1324-avoiding property of p that all entries in 
the top part of a domino in the domino decomposition of p are larger than all 
entries in subsequent dominoes in that decomposition. Similarly, all entries in 
the bottom part of a domino in the domino decomposition of p are larger than 
all entries in the bottom part of subsequent dominoes of that decomposition. 

Also note that the top part of a domino is always 213-avoiding, and the 
bottom part of a domino is always 132-avoiding. 


Example 4.50 
Let p = 3612745. Then a; = 3, a2 = 2, and the sequence of the a; ends here. 


Example 4.51 
Let p = 76285314. Then a, = 7, ag = 6, a3 = 3, a4 = 1 and the sequence of 
the a; ends here. 


Let D,, be the set of dominoes of size n, and let B, be the set of words of 
length n over the alphabet {u, d}. 


LEMMA 4.52 
There exists an injection f : Au; (1324) > D, x By. 
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FIGURE 4.4 
The domino decomposition of p = 76285314. 


PROOF Let p€ Av,,(1324). Take the domino decomposition of p defined 
above. At the same time, create a word w = w1w2--- Wp over {u, d} as follows. 
Go through all entries of p, in decreasing order, from n to 1. Consider the 
domino decomposition of p. If, in that decomposition, n + 1 — 7 goes to the 
top part of a domino, then let w; = u; otherwise, let w; = d. At the end, 
concatenate the top part of all dominoes in the domino decomposition of p 
into a 213-avoiding permutation p’ on the largest entries of [n] preserving 
their pattern, and the bottom parts into a 132-avoiding permutation on the 
smallest entries of [n] preserving their pattern. Finally, create the domino D 
whose top part is p’, whose bottom part is p”, and the horizontal location of 
the entries that belong to p’ is the same as it was in p. 

The obtained domino D and the obtained word over {u,d} form f(p). 

This map f is injective, since p can be recovered from its image (D, w). The 
horizontal position of each entry of p can be recovered from the position of 
the corresponding entry in D, and the vertical position of each entry, starting 
with n and moving in decreasing order, can be recovered from w. 


Example 4.53 
Let p = 76285314 as in Example 4.51, illustrated in Figure 4.4. The entries 
8, 7, 4, and 3 are in top parts of dominoes, so it is the first, second, fifth and 
sixth letters of the word w that will be letters u, implying that w = wudduudd. 
The top entries determine the partial permutation 7834, and the bottom 
entries determine the partial permutation 6251. That leads to a domino D 
whose top part is p’ = 7856, and whose bottom part is p’ = 4231. The entries 
that belong to the top part are in the first, fourth, sixth, and eighth horizontal 
position of D, just as they were in those positions of p. See Figure 4.5 for an 
illustration. 
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FIGURE 4.5 
The domino D, where p = 76285314, and f(p) = (D, uudduudd). 


Example 4.54 

We will show how to recover p from its image (D,w), where D is the domino 
shown in Figure 4.5, and w = uudduudd. The word w shows that entries 
8, 7, 4, and 3 of p are in the top part of their dominoes in the domino 
decomposition of p, while entries 6, 5, 1, and 2 are in the bottom part. In 
order to recover p, the vertical positions of the four entries in the top part 
of D have to be adjusted so that their pattern does not change, but their 
set becomes {3, 4, 7,8}, that is, their sequence becomes the subsequence 7834. 
The horizontal positions of these four entries do not change, they are still in 
the first, fourth, sixth, and eighth positions. Similarly, the vertical positions 
of the remaining four entries must be rearranged so that their pattern does 
not change, but their set becomes {1,2,5,6}, that is, their sequence becomes 
the subsequence 6251, and they are still in the second, third, fifth and seventh 
horizontal positions. This leads to the permutation p = 76285314. 


THEOREM 4.55 
For all positive integers n, the inequality 
Av,,(1324) < 13.5” 
holds. 
PROOF A routine application of Stirling’s formula shows that Lemma 


4.48 implies that d, < 6.75”, and then our claim follows from Lemma 4.52, 
since |B,,| = 2”. 
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4.4.2 Pattern 1342 


In this subsection, we turn our attention to the pattern 1342. Interestingly, we 
will be able to provide an exact formula for Av,,(1342). This is exceptional; 
the only other pattern longer than three for which an exact formula is known 
is 1234. The formula is given by the following theorem. 


THEOREM 4.56 
For all positive integers n, the equality 


Avy (1342) = (—1)""}. la) 
: ni mitt (20-4)! fn —-14+2 
+3) (-1)*: 2 os : ) 


i=2 


holds. 


This is a very surprising result. It is straightforward to prove from this 
formula that Av, (1342) < 8” for all n, and that limp. 7/ AVn (1342) = 8. 

The result itself is not the only interesting aspect of the facts surrounding 
the pattern 1342. We will see that permutations avoiding 1342 are in bijection 
with two different kinds of objects that at first look totally unrelated. The 
first, and for our purposes, more important, type of objects is a specific kind 
of labeled trees. 


DEFINITION 4.57 [127] A rooted plane tree with non-negative integer 
labels €(v) on each of its vertices v is called a 3(0,1)-tree if it satisfies the 
following conditions: 


e if v is a leaf, then €(v) = 0 (this explains the number 0 in the name of 


B(0, 1)-trees), 
e ifv is the root and v4, V2,°-++ , Up are its children, then (v) = ae L(up), 


e if v is an internal node and v1, v2,--+ , Uz are its children, then the in- 
equality (v) <1+ Sa L(v~) holds (this explains the number 1 in the 
name of 3(0, 1)-trees). 


Example 4.58 
Figure 4.6 shows a 6(0,1)-tree on 12 vertices. 


Let us call a permutation p = pip2--++ Pp, skew indecomposable if there exists 
no k € [n— 1] so that for all i < k < j, we have p; > p;. In other words, p 
is skew indecomposable if it cannot be cut into two parts so that everything 
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FIGURE 4.6 
A 8(0,1)-tree. 


before the cut is larger than everything after the cut. For instance, 3142 is 
skew indecomposable, but 43512 is not as we could choose k = 3, that is, we 
could cut between the third and fourth entries. If a permutation is not skew 
indecomposable, then we will call it skew decomposable. 

The importance of 3(0, 1)-trees for us is explained by the following theorem. 


THEOREM 4.59 

For all positive integers n, there is a bijection F from the set of skew in- 
decomposable 1342-avoiding n-permutations to the set pDsOv of 3(0, 1)-trees 
on n vertices. 


Example 4.60 
Let n = 3. Then there are three skew indecomposable n-permutations, 123, 
132, and 213, and they all avoid 1342. Correspondingly, there are indeed three 
G(0, 1)-trees on three vertices, as can be seen in Figure 4.7. 

Let tn = |DROv 
crucial step forward as it is known [127] that 


. If we can prove Theorem 4.59, then we have made a 


(2n — 2)! 


ie a eee 
(n+ 1)!(n — 1)! 


(4.6) 


We start by treating two special types of (0, 1)-trees on n vertices. There 
are two things that contribute to the structure of a (0, 1)-tree, namely its (un- 
labeled) tree structure, and its labels. We will therefore first look at 6(0,1)- 
trees in which one of these two ingredients is trivial, that is, 6(0, 1)-trees that 
consist of a single path only, and 6(0,1)-trees in which all labels are zero. 
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1@ 0@ 


0@ 0@ 0@ 0 


FIGURE 4.7 
The three 6(0, 1)-trees on three vertices. 


LEMMA 4.61 

There is a bijection f from the set of 1342-avoiding n-permutations starting 
with the entry 1 and the set of 8(0,1)-trees on n vertices consisting of one 
single path. 


Note that a simpler description of the domain of f is that it is the set of 
231-avoiding permutations of the set {2,3,4--- ,n}. 


PROOF ~ Let p= pipe::-py be an 1342-avoiding n-permutation so that 
pi = 1. Take an unlabeled tree on n nodes consisting of a single path and 
give the label £(i) to its ith node (1 < i < n—1) by the following rule: 


| {j <i so that p; > ps for at least one s >7,}| ifi<n 
l(t) = 
€(n—1) ifi=n. 


That is, €(i) is the number of entries weakly on the left of p; which are 
larger than at least one entry on the right of p;. Note that this way we could 
define f on the set of all n-permutations starting with the entry 1, but in that 
case, f would not be a bijection. (For example, the images of 1342 and 1432 
would be identical.) 


Example 4.62 

If p = 143265, then the labels of the nodes of f(p) are, from the leaf to the 
root, 0,1,2,0,1,1. See Figure 4.8. For easy reference, we wrote p; to the ith 
node of the path f(p). To avoid confusion, in this figure, and for the rest of 
this subsection, the entries of p will be written in small, Roman letters, and 
the labels of the nodes will be written in large italic letters. 
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FIGURE 4.8 
The (0, 1)-tree of p = 143265. 


It is easy to see that f indeed maps into the set of 6(0,1)-trees: €(i +1) < 
€(i) +1 for all 2 because there can be at most one entry counted by ¢(¢ + 1) 
and not counted by ¢(i), namely the entry p;,1. All labels are certainly non- 
negative and é(1) = 0. 

To prove that f is a bijection, it suffices to show that it has an inverse, 
that is, for any 6(0,1)-tree T consisting of a single path, we can find the only 
permutation p so that f(p) = T. We claim that given T, we can recover the 
entry n of the preimage p. First note that p is 1342-avoiding and starts with 
1, so any entry on the left of n must be smaller than any entry on the right 
of n. In particular, the node preceding n must have label 0. Moreover, as n 
is larger than any entry following it in p, the entry n is the leftmost entry 
p; of p so that €(j) > O for all 7 > 7 if there is such an entry at all, and 
n = py if there is none. That is, n corresponds to the node that starts the 
uninterrupted sequence of strictly positive labels that ends in the last node as 
long as there is such a sequence. Otherwise, n corresponds to the last node. 

Once we locate where n is in p, we can simply delete the node corresponding 
to it from T and decrement all labels after it by 1. (If this means deleting 
the last node, we just change ¢(n — 1) so it is equal to 0(n — 2) to satisfy the 
root-condition.) We can indeed do this because the node preceding n had 
label 0 and the node after n had a positive label (1 or 2), by our algorithm 
to locate n. Then we can proceed recursively, by finding the position of the 
entries n — 1,n — 2,---,1 in p. This clearly defines the inverse of f, so we 
have proved that f is a bijection. 
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As we promised, we continue by explaining which skew indecomposable 
1342-avoiding permutations correspond to 6(0, 1)-trees in which all labels are 
equal to zero. 


LEMMA 4.63 
There is a buyection g from the set of 132-avoiding n-permutations ending 
with n to the set of 3(0,1)-trees on n vertices with all labels equal to zero. 


Note that we could describe the domain of g as the set of skew indecompos- 
able 132-avoiding n-permutations, or as the set of (n — 1)-permutations that 
avoid 132. 


PROOF In this proof, we can obviously think of our ((0,1)-trees as 
unlabeled rooted plane trees. A branch of a rooted tree is a subtree whose 
root is one of the root’s children. Some rooted trees may have only one branch, 
which does not necessarily mean they consist of a single path. 

We will construct g inductively. There is only one unlabeled ((0, 1)-tree on 2 
vertices and it is the image of the only 1-permutation; p = 1. Using induction, 
suppose we have already constructed g for all positive integers k < n. Let 
p be a 132-avoiding permutation of length n. Let p! = pipo-+:Pn—1. Then 
there are two possibilities. 


(a) The first case is when p’ is skew decomposable, that is, we can cut p’ 
into several (at least two) strings pi1),p(2),--- , Pin) SO that all entries 
of pi) are larger than all entries of pi;) if i(j7. In this case, g(p) will 
have h branches, the branch 6; satisfying g(p,;)) = bi. We then obtain 
g(p) by connecting all branches b; to a common root. Given that we are 
in a 8(0,1)-tree, the label of the root is determined by the labels of its 
children. 


(b) The second case is when p’ is skew indecomposable. As p avoids 132, 
this is equivalent to saying that p’ ends with its maximal entry n — 1. 
In this case, g(p) will have just one branch 0, that is, the root of g(p) 
will have only one child. We define b; = g(p’). 


Again, we prove that g is a bijection by showing that it has an inverse. 
Let T be an unlabeled plane tree on n vertices with root q. Let q have 
t children, and say that, going left-to-right, they are roots of the branches 
bi, b2,--- , bz, which have nj, 2,--- ,nz nodes. Then by induction, for each 7, 
the branch b; corresponds to a 132-avoiding n;-permutation ending with n,. 
Now add ee 41”; to all entries of the permutation p; associated with 6;, 
then concatenate all these strings and add n to the end to get the permutation 
p associated with T. 

It is straightforward to check that this procedure always returns the original 
permutation, proving our claim. 
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FIGURE 4.9 
The (0, 1)-tree of p = 45631278. 


Example 4.64 

The permutation 45631278 corresponds to the 8(0,1)-tree with all labels 
equal to 0 shown in Figure 4.9. For easy reference, we write p, to the root of 
g(p), and proceed analogously for the other entries in the recursively defined 
subtrees. 


An easy way to read off the corresponding permutation once we have its 
entries written to the corresponding nodes is the well-known postorder read- 
ing: for every node, first write down the entries associated with its children 
from left to right, then the entry associated with the node itself, and do this 
recursively for all the children of the node. 

The reader is invited to prove that if p is as in Lemma 4.63 and the entries 
of p are associated to the vertices of g(p) as in Example 4.64, vertex i will be 
a descendant of vertex j if and only if 2 is on the left of 7 in p andi < 7. 

Our plan is to bring Lemmas 4.61 and 4.63 together to prove Theorem 
4.59. This needs some preparation. Optimally, we would take a 1342-avoiding 
skew indecomposable n-permutation p, associate its entries to the nodes of an 
unlabeled plane tree T, then define the labels of this tree so that it becomes 
a 6(0,1)-tree. The question is, however, how do we know what T we should 
use, and if T is given, how do we know in what order we should write the 
entries of p to the nodes of IT’? In what follows, we develop the notions to 
decide these questions. 


DEFINITION 4.65 Two n-permutations x and y are said to have the 
same weak type if the left-to-right minima of x are the same as those of y, 
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and they are in the same positions. 


Example 4.66 

Permutations 456312 and 465312 have the same weak type since their left- 
to-right minima are 4, 3, and 1, and they are located at the same positions. 
Permutations 31524 and 34152 do not have the same weak type. 


PROPOSITION 4.67 
Each nonempty weak type has exactly one 182-avoiding permutation. 


PROOF Take all entries that are not left-to-right minima and fill all 
empty positions between the left-to-right minima with them as follows: In 
each step place the smallest entry that has not been placed yet that is larger 
than the previous left-to-right minimum. (This is just what we did in the 
proof of the Simion-Schmidt bijection in Lemma 4.4.) 

On the other hand, the resulting permutation will be the only 132-avoiding 
permutation of this weak type because any time we deviate from this proce- 
dure, that is, we place something else, not the smallest such entry, we create 
a 132-pattern. 


DEFINITION 4.68 The normalization N(p) of an n-permutation p is 
the only 182-avoiding permutation having the same week type as p. 


Example 4.69 
If p = 356214, then N(p) = 345216. J 


DEFINITION 4.70 The normalization N(T) of a G(0,1)-tree T is the 
8(0,1)-tree which is isomorphic to T as a plane tree, with all labels equal to 
zero. 


It turns out that normalization preserves the skew indecomposable property. 


PROPOSITION 4.71 
A permutation p is skew indecomposable if and only if N(p) is skew indecom- 
posable. 


PROOF (The author is grateful to Aaron Robertson, who found and 
corrected a mistake in his original argument.) We will show that whether p is 
skew decomposable or not depends only on the set and position of its left-to- 
right minima, which is obviously equivalent to the claim to be proved. Let C 
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be set of permutations having the same weak type as p, given by the set and 
position of its left-to-right minima. It is clear that if p € C is decomposable, 
then the only way to cut it into two parts (so that everything before the cut 
is larger than everything after the cut) is to cut it immediately on the left 
of a left-to-right minimum a < n. Now if there is a left-to-right minimum 
in position n — a+ 2, then the entries 1,2,--- ,a@— 1 must occupy positions 
n-a+2,n—a+l1,---,n. Therefore, we can cut immediately on the left of 
position n — a+ 2, and p is decomposable. 

If there is no such a, then for all left-to-right minima m, all the entries 
1,2,--- ,m—1 must be to the right of m. However, in one of the positions 
n—-m+2,n—m-+1,---,n, there exists an element y > m, implying that our 
permutation p is not decomposable. 


COROLLARY 4.72 
If p is a skew indecomposable n-permutation, then N(p) always ends in the 
entry n. 


PROOF Note that the only way for a 132-avoiding n-permutation to 
be skew indecomposable is for it to end with n. If p is a 132-avoiding n- 
permutation and n is not the last entry, then we may cut p immediately after 
the entry n. Then the statement follows from Proposition 4.71. 


Now we are in a position to prove Theorem 4.59. 


PROOF (of Theorem 4.59.) Let p be a skew indecomposable 1342- 
avoiding n-permutation. Take N(p) = r. By Corollary 4.72 its last element 
is n. Define F(r) to be the unlabeled plane tree S associated with r by the 
bijection g of Lemma 4.63. So g is just the restriction of F' to the set of skew 
indecomposable 132-avoiding permutations. 

This unlabeled tree S is the tree we are going to work with. First, we will 
write the entries of p to the nodes of S. (The reader should recall that we 
did this in the proof of Lemma 4.61, and that the entries of p written to the 
nodes of S are not to be confused with the labels of the nodes.) We will do 
this in the order specified by p and N(p). That is, N(p) is a 132-avoiding 
permutation, so its entries are in natural bijection with the nodes of S as 
we saw in Lemma 4.63. We then let the permutation p(N(p))~! act on the 
entries of N(p) (written to the nodes of S') to get the order in which we write 
the entries of p to S. Note in particular that the left-to-right minima are kept 
fixed. 


Example 4.73 
Let p = 4621357. Then N(p) = 4521367, and the unlabeled plane tree S$ 
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e 2@ 


FIGURE 4.10 
The unlabeled trees of N(p) = 4521367 and p = 4621357. 


associated with these permutations is shown in Figure 4.10, together with the 
order in which the entries of p are written to the nodes. Note that p and N(p) 
only differ in the transposition Oo This is why it is these two entries whose 
positions have been swapped. 


Now we are going to define the label ¢(v) of each node v for the new {(0, 1)- 
tree T = F(p) that we are constructing from S. As an unlabeled tree, T will 
be isomorphic to S, but its labels will be different. Let v; be the ith node of 
T in the postorder reading, the node to which we associate p;. We say that 
the entry p; beats the entry p; if there is an entry pp so that pp, pi,p; are 
written in this order (so h <i < j) and they form a 132-pattern. Moreover, 
we say that p; reaches p, if there is a subsequence pj, Dita,,°**DPita:,Pk Of 
entries so that i <<i+a, <i+ag<-+-:-<i+a, <k and that each entry in 
this subsequence beats the next one. In particular, if x beats y, then x also 
reaches y. 


Example 4.74 
Let p = 3716254. Then 7 beats 6, 6 beats 2; therefore, 7 reaches 2. 


Finally, we define the label @(v;) of the non-root vertex v; of T = Fp) to 
which the entry p; is associated as follows. The label ¢(v;) is the number of 
descendants v; of v; (including v; itself) for which there exists at least one 
index k >7 so that the entry p; reaches the entry p x. 

Let F'(p) be the ((0, 1)-tree defined by these labels. Note that a descendant 
of 7 is an element of the subtree rooted at 7. Note that this rule is an extension 
of the labeling rule we have in Lemma 4.61. 

First, it is easy to see that F indeed maps into the set pROv. Indeed, 
let v be an internal node and let v1, v2,---+ ,v~% be its children. Then @(v) < 
eS Seat (vz) because there can be at most one entry counted by é(v) and not 
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FIGURE 4.11 
The image F'(p) of p = 4621357. 
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FIGURE 4.12 


The tree S and F(p) for p = 58371624. 


counted by any of its children’s labels, namely v itself. Second, all labels are 
certainly non-negative and all leaves, that is, the left-to-right minima, have 
label 0. 


Example 4.75 

In Example 4.73 we have created the unlabeled tree S for p = 4621357. Ap- 
plication of the above rule shows that F'(p) is the G(0, 1)-tree shown in Figure 
4.11. Indeed, the only 132-pattern of p is 465, and that is counted only once, 
at the entry 6. 


Example 4.76 

Let p = 58371624, then we have N(p) = 56341278, giving rise to the unlabeled 
tree shown on the left of Figure 4.12. We then compute the labels of Fp) to 
obtain the tree shown on the right of Figure 4.12. 
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To prove that F is a bijection, it suffices to show that it has an inverse. 
That is, it suffices to show that for any 6(0,1)-tree T € DEO, we can find 
a unique permutation p so that F'(p) = T. 

Before we explain why F' has an inverse, we prove two propositions that 
show a simple way to find the position of the entry n in p if F(p) is given. 


PROPOSITION 4.77 

Suppose pn # n, that is, n is not associated with the root vertex. Then each 
ancestor of n, including n itself, has a positive label. If pn =n, then l(u,) = 0 
and thus there is no vertex with the above property. 


PROOF _ Ifp, =n, then there is nothing on the right of n to reach, thus 
€(vp,) enumerates an empty set, yielding ¢(v,) = 0. Suppose now that p, is 
not the root vertex. We will show that the entry n always reaches p,,, which 
implies the statement of the proposition. 

Note that n 4 p,, because then p would be skew decomposable. Let a, 
be the smallest entry on the left of n. Let n1 be the rightmost entry that n 
beats; in other words, the rightmost entry that is still larger than ay. 

If n, = pn, we are done; otherwise, let az be the smallest entry on the left 
of n,, and let ng be the rightmost entry that n, beats; in other words, the 
rightmost entry that is still larger than ag. If n2 = py, we are done; otherwise, 
continue this same process. We claim that this process eventually reaches the 
rightmost entry pn. 

Indeed, how could the process get stuck? In the first step, if ny cannot be 
defined, that means that all entries on the right of n are smaller than all entries 
on the left of n, which means that p is skew decomposable, contradicting our 
assumptions. Then, in the second step, az is always well-defined, and if ng 
does not exist, that means that all entries on the right of n; are smaller than 
ag, (they cannot be larger than a; because of the definition of n;), while all 
entries on the left of n, are, by the definition of az, at least as large as ag. 
This again implies that p is skew decomposable, which is a contradiction. 

Continuing in this way, we see that the process will end only when the 
rightmost entry py, is reached and p, = nz for some k. As n beats n 1, then 
n 1 beats ng, and_so on, n reaches all the n;, including p, = ng, and our proof 
is complete. 


The only problem is that there could be many vertices with the property 
that all their ancestors have a positive label. If that happens, we resort to the 
following proposition to locate the vertex associated with n. 


PROPOSITION 4.78 
Suppose pp, #n. Then n is the leftmost entry of p that has the property that 
each of its ancestors (including itself) has a positive label. 
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PROOF Suppose pz and n both have this property and that pz, is on 
the left of n. (If there are several candidates for the role of py, choose the 
rightmost one). If pg, beats an element y on the right of n by participating in 
the 132-pattern xp, y, then xp, ny is a 1342-pattern, which is a contradiction. 
So px does not beat such an element y. In other words, all elements after n are 
smaller than all elements before pz. Still, p, must reach elements on the right 
of n, thus it beats some element v between pz, and n. This element v in turn 
beats some element w on the right of n by participating in some 132-pattern 
tuw. However, this would imply that tunw is a 1342-pattern, a contradiction, 
which proves our claim. 


Now we are ready to prove that F is a bijection. The author is grateful 
to Julika Mehrgardt for valuable discussions on this proof. We prove our 
statement by induction on n, the case of n = 1 being obvious. Let us assume 
that we have already proved the statement for all positive integers less than n. 
With a slight abuse of notation, we denote the bijections belonging to these 
smaller integers also by F’. Let T be a 3(0,1)-tree on n vertices. Note that T, 
even without its labels, specifies the weak class of any purported permutation 
p for which F'(p) = T. Let p; be the leftmost entry (and so, the leftmost 
left-to-right minimum) of p, determined from T. Let v; be the leftmost leaf 
of T. We consider the following four cases. 


1. Let us assume that v; is not the only child of its parent x. In this 
case, we can simply remove v; from T. The remaining tree T” is a valid 
8(0,1)-tree that is smaller than T, so by our induction hypothesis, it 
has a unique preimage F~!(T’) = p’. Now prepend p’ by the entry p1 
(and add 1 to all entries in p’ that are larger than or equal to p;) to get 
the unique preimage F~!(T’) = p. Note that the addition of p; to the 
front of p’ does not create a 1342-pattern since p, is larger than the first 
entry of p’ in this case. 


2. Let us assume that v; is the only child of its parent x, but ¢(7) = 0. In 
this case, we can still remove v; from JT and proceed as in the previous 
case. Note that in this case, the entry associated to x will be p; + 1. 


3. Now let us assume that v; is the only child of x, and while ¢(x) 4 0, 
there exists an ancestor of x that has label 0. Let v be the closest such 
ancestor of xz. That means that v is not the root of T, since the only 
way for the root to have label 0 is by all of its children to have label 0. 


Now temporarily remove the subtree T,, rooted at v from T except for 
its root. In other words, shrink T, to its root v. Let T’ be the remaining 
tree. Note that since ¢(v) = 0, the tree T’ is a valid 6(0,1)-tree . As T’ 
is smaller than T, it has a unique preimage p’ under F' by our induction 
hypothesis. We claim that there is exactly one possibility for the set of 
entries of p that are associated to the vertices of T,,, and there is exactly 
one possibility for the partial permutation p, of these entries. 


In Any Way but This. Pattern Avoidance. The Basics. 185 


For the first claim, note that the set S of entries associated to the ver- 
tices of T, is equal to the set Z of entries associated to the vertices of 
Ty by N(p). Recall from the proof of Proposition 4.67 that the normal- 
ization procedure keeps the left-to-right minima of a permutation fixed, 
and then puts the smallest eligible entry into each available slot, pro- 
ceeding from left to right. (This is the only way to get a 132-avoiding 
permutation in the given weak class.) If S 4 Z held, that would mean 
that this procedure was not followed, and hence, a 132-pattern starting 
in T, but ending outside T,, was formed, contradicting the assumption 
f(v) = 0. 

For the second claim, note that T,, becomes a (0, 1)-tree if we change 
the label of its root v from 0 to the sum of the label of its children 
(which does not change anything since neither the label 0 nor the new 
label carries any new information for T,,). As T, is smaller than T, it 
has a unique preimage p, under F’ by our induction hypothesis. 


The partial permutations p,, p’, and the set S on which p, is taken 
together determine the unique permutation F'~!(p). 


4. Finally, let us assume that v is the only child of x, and no ancestor 
of x has zero for its label, that is, all vertices on the path from x to 
the root have positive labels. By Proposition 4.78, that means that x is 
the vertex to which the entry n is associated. In other words, n is the 
second entry of our purported permutation p. Now remove x from T’ 
(that is, connect v; directly to the parent of 2, and decrease the labels 
of each ancestor of « by 1). This can be done since all those labels were 
positive. Let T’ be the obtained tree. Construct the unique permutation 
p' = f—'(T) whose existence is guaranteed by our induction hypothesis. 
Insert n to its second position to get F~! = p. This certainly does not 
create a 1342-pattern, since n could only play the role of 4 in such a 
pattern, but there is only one entry preceding n in p. 


This proves that F is a bijection, completing the proof of Theorem 4.59. 


COROLLARY 4.79 


The number of skew indecomposable 1842-avoiding n-permutations is 


(2n — 2)! 


DIOD) = 4, =3.9n-2. A 
Pa (n+ 1)\(n — 1)! 


(4.7) 


PROOF _ Follows from (4.6) and Theorem 4.59. Il 


Computing the numbers Av,,(1342) is now a straightforward task using 
generating functions. 
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LEMMA 4.80 
Let H(z) = 7) p Avn(1342)2". Furthermore, let F(z) = ya tne”, that 
is, let F(z) be the generating function of the numbers of skew indecomposable 


1342-avoiding permutations. Then 


es re 322 
ee oe ©) - T—F@ ~ 82 s20e+1-a-ape =) 


PROOF _ Tutte [313] has computed the ordinary generating function of 
the numbers t¢,, and obtained 


— n 7 n—-1 (2n — 2)! n 
_ _ ee 4. 
F(z) da tne ee 2 Ga Din Dl (4.9) 
_ 2 4+122-1+4 (1 82)3/? (4.10) 
7 32z 


The coefficients of F(z) are the numbers of skew indecomposable 1342- 
avoiding n-permutations. Any 1342-avoiding permutation has a unique de- 
composition into skew indecomposable permutations. This can consist of 
1,2,-++ ,n blocks, implying that Av,,(1342) = S77, ti8n—;. Therefore, H(z) = 
1/(1 — F(z)) as claimed. 


It is time that we mentioned the other kind of objects that are in bijection 
with these permutations. These are rooted bicubic maps, that is, planar maps 
in which each vertex has degree three, there is a distinguished half-edge (the 
root), and the underlying graph is bipartite. Tutte was enumerating these 
maps (according to the number 2(n+ 1) of vertices) when he obtained formula 
(4.9), and Cori, Jacquard, and Schaeffer [127] then used the 6(0, 1)-trees to 
find a more combinatorial proof of Tutte’s result. 

In Chapter 8, we will mention another surprising occurrence of the numbers 
Av, (1342). 

Now that we have the generating function of the numbers Av,,(1342), we 
are in a position to obtain an explicit formula for their number. That formula 
will prove Theorem 4.56. 


PROOF (of Theorem 4.56). Multiply both the numerator and the de- 
nominator of H(z) by (—82z? + 20z + 1) + (1 — 8z)°/?. After simplifying, we 
get 
(1 —8z)8/? — 827 + 202 +1 


= 4.11 
ae Az+ 3 ea) 
As (1—82)9/? = 1-122 + 0593-2" 2" CHE formula (4.11) implies our 


claim. | 
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So the first few values of Av,,(1342) are 1, 2, 6, 23, 103, 512, 2740, 15485, 
91245, 555662. We mention that Jonathan Bloom and Sergi Elizalde [41] 
found a different proof for Theorem 4.56 in 2013. 

In particular, one sees easily that the formula for Av,,(1342) given by The- 
orem (4.56) is dominated by the last summand; in fact, the alternation in 
sign assures that this last summand is larger than the whole right hand side if 
n> 8. As ieee < “ by Stirling’s formula, we have proved the following 
exponential upper bound for Av, (1342). 


COROLLARY 4.81 
For all n, we have Av,,(1342) < 8”. 


On the other hand, it is routine to verify that the numbers t, satisfy the 
recurrence t, = (8n — 12)tn-1/(n + 1). As we explained immediately af- 
ter Conjecture 4.15, the fact that Av,(1342) < 8” implies that the limit 
L(1342) = limyn-+o0. */ Av,,(1342) exists. Therefore, by the Squeeze Principle, 
we obtain the following Corollary. 


COROLLARY 4.82 
The equality 
L(1342) = 8 


holds. 


This result was striking for two different reasons. On one hand, this is the 
third time that the exact value of L(q) = limp. */Avn(q) for some pattern 
q was computed. Indeed, we have seen that 


4 if q is of length 8, 


lim L(q) = ¢ (k—1)? if ¢=123---k, 


noo 
8 if g = 1342, 


In other words, in every case when we saw an exact answer, the exact 
answer was an integer. In general, however, that does not hold. It is proved 
in [57] that L(12453) = 9+ 4/2. So these limits are not even always rational. 
Articles [57] and [60] contain other examples for patterns gq for which L(q) is 
irrational. However, in all cases when L(q) is known, it is a real number of 
the form a + by/2, where a and b are integers. 

The other surprise provided by Corollary 4.82 is that 


L (1342) = 8 # L(1234) = 9. 


That is, even in the logarithmic sense, the sequences Av, (1342) and Av,,(1234) 
are different. This phenomenon is not well understood. If we could under- 
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stand why the pattern 1342 is really so difficult to avoid, then maybe we could 
use that information to find other, longer patterns that are difficult to avoid. 

As far as the fact that D(1324) > £(1234) goes, that is not an accident 
in the following sense. A pattern q is called layered if q consists of a union 
of decreasing subsequences (the layers) so that the entries increase from one 
subsequence to the next. For instance, g = 3215476 is a layered pattern. It is 
proved in [57] that if q is a layered pattern of length k, then 


L(q) > L(12-+-k) = (k—1)?. 


We will formally define layered patterns in the next section. 


4.4.3 Pattern 1234 


The pattern 1234 is a monotone pattern; therefore Theorem 4.22, which pro- 
vides an asymptotic formula and a very good upper bound for the numbers 
Av,,(123---k), applies to it. However, we would like to point out that using 
certain techniques beyond the scope of this book, Ira Gessel [191] proved the 
following exact formula for these numbers 


7 (2k\ (n\? 3k? 4+ 2k +1—n—2nk 
Av, (1234) = 2- or 4.12 
vette) He (k+ 1)2(k+2)(n—k+1) oe) 
The alert reader has probably noticed that the summands on the right-hand 
side are not always non-negative, which decreases the hopes for a combinato- 


rial proof. However, a few years later Gessel found the following alternative 
form for his formula [190] 


Avn (1234) = EscEs > @ (; : i) . 7 a; (4.13) 


k=0 
In this new form, all terms are non-negative, but there is still a division, 
suggesting that a direct combinatorial proof is probably difficult to find. 
We will return to the surprising complexity of Gessel’s formulae in the next 
chapter. 


Dn 


4.5 Proof of the Stanley-Wilf Conjecture 


The goal of this section is to present a proof of the Stanley—Wilf conjecture. 
In order to achieve this goal, we will discuss one more conjecture involving 
0-1 matrices, show why it implies the Stanley—Wilf conjecture, and then prove 
the conjecture on the 0-1 matrices. The proof we present was announced by 
Adam Marcus and Gabor Tardos in the Fall of 2003. Before that, Noga Alon 
and Ehud Friedgut [12] proved a somewhat weaker result. 
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4.5.1 Fiiredi-Hajnal Conjecture 


Let us extend the notion of pattern avoidance to 0-1 matrices as follows. 


DEFINITION 4.83 Let A and P be matrices whose entries are either 
equal to 0 or to 1, and let P be of sizek x £. We say that A contains P if A 
has ak x € submatrix Q so that if Pi; =1, then Qi; =1, for alli and j. If 
A does not contain P, then we say that A avoids P. 


In other words, A contains P if we can delete some rows and some columns 
from A and obtain a matrix Q that has the same shape as P, and has a 1 in 
each position P does. Note that Q can have more 1 entries than P, but not 
less. Note that if all entries of A are equal to 1, then A contains all matrices 
P that have shorter side lengths than A. 


Example 4.84 


1001 
0110 00 : 

Let A= 0011 , and let P = cae Then A contains P as can be seen 
1000 


by deleting everything from A except the intersection of the first and third 


: a then A avoids Q. 


rows with the third and fourth columns. If Q = ie l 


O 


All matrices in this section will be 0-1 matrices, so we will not 
repeat that condition any more. The famous Fiiredi—Hajnal conjecture, 
which was originally stated as a question, sounds similar to the Stanley—Wilf 
conjecture. 


CONJECTURE 4.85 [186] [Ftiredi-Hajnal conjecture] Let P be any 
permutation matrix, and let f(n, P) be the maximum number of 1 entries that 
a P-avoiding n x n matrix A can have. Then there exists a constant cp so 
that 

f(n, P) < cpn. (4.14) 


We point out that the condition that P is a permutation matriz is important. 
See Exercise 43 for a non-permutation matrix for which (4.14) does not hold. 


4.5.2 Avoiding Matrices versus Avoiding Permutations 


In this scection, we present an argument proving that the Fiiredi-Hajnal 
conjecture implies the Stanley—Wilf conjecture. The original form of the 
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argument is due to Martin Klazar. The simplified form of the argument that 
we present is due to Doron Zeilberger and Vincent Vatter. 

Let q be a pattern of length k, and let p be an n-permutation that avoids 
q. Then the permutation matrix of p avoids the permutation matrix By of q. 
Now let M,,(q) be the number of n x n matrices with 0 and 1 entries that avoid 
B,. Then Av,,(q) < M;(q). We will now find an upper bound for M,,(q). 

If Conjecture 4.85 is true, then we know that there exists a constant b so 
that f(n, By) < bn. Let A be an n x n matrix that avoids By, and let us cut 
A up into 2 x 2 blocks. (There will be smaller blocks at the end if n is odd.) 
If a block contains no 1s, then replace that block with a 0; if it contains at 
least one 1, then replace that block with a 1. 

This results in a matrix of size [n/2] x [n/2] that is also B,-avoiding, and 
therefore has at most f([n/2], By) < 6[n/2] entries equal to 1. Furthermore, 
each of these ones comes from a nonzero 2x 2 block of A. There are 24—1 = 15 
possibilities for each such block. Therefore, the number M,,(q) of all possible 
matrices A we could have started with satisfies 


Mn(q) < 159/21. Men joy (q)- 


Repeat this argument until the right-hand side becomes M1(q) = 2. Then we 
get 
Avn(q) < Mn(q) < 157". (4.15) 


Therefore, we have proved the following. 


PROPOSITION 4.86 
[226] If the Fiiredi—Hajnal conjecture is true, then the Stanley-Wilf conjecture 
is also true. 


4.5.3 Proof of the Fiiredi-Hajnal Conjecture 


We close this chapter by presenting the spectacular proof of Conjecture 4.85, 
given by Adam Marcus and Gabor Tardos in 2003 [250]. 

Let P be a k x k permutation matrix, and let A be an n x n matrix that 
avoids P and contains exactly f(n,P) entries 1 (as defined in Conjecture 
4.85). Let us assume for simplicity that n is divisible by k?. The crucial idea 
of the proof is a decomposition of A into blocks. While the simple idea of 
decomposing a matrix into smaller matrices is not new, the novelty of the 
Marcus~Tardos method is that it decomposes A into 7 - z blocks, which are 
each of size k? x k?. 

For (i,j) € [fz] x [#5], let Sj; denote the submatrix (block) of A that 
consists of the intersection of rows (i — 1)k? + 1, (i — 1)k? + 2,--- ,ik? and 
columns (j — 1)k? + 1,(j — 1)k? + 2,--- ,jk?. We will now contract A into 
a much smaller matrix B as follows. Each entry of B will contain some 
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information about a block of A. The matrix B = (b;,;) is of size & x 4, and 


0 if all entries of S;,; are zero, 
bij = 
1 if not all entries of S;,; are zero. 


PROPOSITION 4.87 
The matrix B avoids P. 


PROOF Let us assume that B contains P, and let us take a copy PR 
of P in B. Then considering A, and using the fact that P is a permutation 
matrix, we can take a 1 from each block of A that defined an entry of P., and 
get a copy of P in A. 


The next crucial step is the following definition. 


DEFINITION 4.88 A block S;; of A is called wide if it contains a 1 
in at least k different columns. Similarly, a block is called tall if it contains a 
1 in at least k rows. 


Note that a block has k? columns, but it is called wide if at least k of these 
columns contain a 1. 


LEMMA 4.89 
For any fixed j, the set of blocks Cj; = {Si,j\|1 <i < ge} of the matrix A 


contains less than (k — i) +1 wide blocks. 


PROOF ~ We show that if the statement of the lemma were false, then A 
would contain a copy of P. Indeed, assuming that the number of wide blocks 
in C; is at least (k — y(*) +1, by the pigeon-hole principle there would be 
a k-tuple of integers 1 < ce, < co < +--+ < cy < k? so that there are k blocks 
Say ,j1 9a9,j0°** » Sax,j that all contain a 1 entry in column c;, for 1 <i < k. 
In that case, it is easy to find a copy of P in A, which is a contradiction. 
Indeed, if the single 1 in column i of P is in row p(t), then choose a 1 from 
column c; of Sq,,,),j- As the blocks Sa,,j,Saz,j,°** »Sa,,j are positioned in a 
column, the n entries 1 chosen this way will prove that A contains P, which 
is a contradiction. 


It goes without saying that the same argument can be made for the array 
of blocks Rj = {S;,;|1 < j < 4}, thereby giving the following lemma. 
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LEMMA 4.90 
For any fixed i, the set of blocks Ri = {Sij|1 < j < ~} of the matric A 


contains less than (k — 1)(*) +1 tall blocks. 


We have seen that A cannot have too many wide or tall blocks, and Propo- 
sition 4.87 seems to suggest that A cannot have too many nonzero blocks 
either. Putting together these observations, we get the following recursive 
estimate. 


LEMMA 4.91 
For any k x k permutation matric P, and any positive multiples n of k?, we 
have 


f(n, P) < (k—1)*f (5?) 4+ 2k3 ("Jn 


PROOF By Proposition 4.87, the number of nonzero blocks is at most 
2 
f (7. P). By Lemmas 4.89 and 4.90, there are at most #(k — 1) wide 


blocks, and at most ~(k — 1)(*) tall blocks. 

Let us count how many 1 entries the various blocks of A can contribute to 
f(n,P). A wide (or tall) block can contribute at most k*+ entries 1, so the 
total contribution of these blocks is at most 


n k? 4 3 k? 
otra o(!) a <ate()n 
If a block is neither wide nor tall, then by the pigeon-hole principle, it can 


contain at most (k — 1)? entries 1. Multiplying this bound by the number of 
nonzero blocks yields that the contribution of all blocks that are not tall or 


wide is at most es 
(k-1°F(4,P). 
Adding all the contributions of all these blocks, the lemma is proved. | 


We now have all necessary tools to prove the Fiiredi-Hajnal conjecture. 


THEOREM 4.92 


For all permutation matrices P of size k x k, we have 


k? 
f(n,P)< 2it(7 


PROOF We prove the statement by induction on n, the initial case of 
n = 1 being obvious. (In fact, the statement is obvious when n < k?, because 
then A has at most k* entries.) 
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Now assume the statement is true for all positive integers less than n, and 
prove it for n. Let n’ = [n/k?|k?. Then by Lemma 4.91, we have 


f(n,P) < f(n’, P) + 2k?n 
n! Ns 
<(k-1)?f (=P) +2n(4 Jn + 2k?n 


as in the worst case, we fill the part of A that cannot be partitioned into 
k? x k? blocks by entries 1. Applying the induction hypothesis to f (z. P), 
we get 


f(n, P) < (k—1)? [an a + 2k? @r + 2k?n 


2 
< 22((k— 1)? +k+1)(7, 
2 
< 2Ki(1, 


since k? > (k—1)2+k+1wheneverk>2. Tf 


The proof of the Stanley—Wilf conjecture is now obvious. We include it 
because we want to show the numerical result obtained. 


COROLLARY 4.93 
For any permutation pattern q of length k, we have 


Avn(q) <¢ 


nm 

q? 
4(k? 

where Cq = 15?* (i), 


PROOF Immediate from Theorem 4.92 and (4.15). 


2 
The value cy = 152**("e) is certainly very high. For instance, if & = 3, then 


we get c, = 1527216 while we have seen in Corollary 4.8 that c, = 4 is the best 
possible result. Josef Cibulka [118] improved the upper bound of Corollary 
4.93 as follows. Let cy be the smallest constant satisfying Av, (q) < cf for all 
patterns q of length k, and all positive integers n. Then there is a constant C 
so that the inequality 

Ck < 9Ck log k (4.16) 


holds. Though this is a significant improvement, the upper bound obtained 
for cy, is still an exponential function of k. 
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FIGURE 4.13 
The diagram of a generic layered permutation. 


The following definition enables us to discuss a disproved conjecture that 
suggested a fairly low (polynomial) value for cx. 


DEFINITION 4.94 A permutation q is called layered if q can be written 
as the concatenation q1q2°*:Qm of the strings qi, for 1 <i<m, where 


(a) each q; is a decreasing sequence of consecutive integers, and 


b) the leading entry of q 1s smaller than the leading entry of qi41, for 
g y 
1<i<m-l. 


In other words, a permutation is layered if it consists of decreasing subse- 
quences (layers) so that the entries decrease within each layer but even the 
largest entry of a layer is smaller than the smaller entry of the next layer. In 
Exercise 13, of the next chapter you are asked to characterize layered permu- 
tations by pattern avoidance. 


Example 4.95 
The patterns 1432765 and 4321576 are both layered. 


See Figure 4.13 for an illustration. 
In [119], the authors use generalized versions of the methods we presented 
in the proof of Theorem 4.39 to show the following. 
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THEOREM 4.96 
Let q be a layered permutation pattern of length k. Then the inequality 


Avn(q) < (4k?)” = (2k)"" 
holds. 


We will give the proof of Theorem 4.96 in Problems Plus 21 and 22. 

Note that numerical evidence suggests that for small values of k, among all 
permutation patterns of length k, layered patterns are the easiest to avoid. 
Therefore, Theorem 4.96 suggested the following conjecture, published in 
[119]. 


CONJECTURE 4.97 Let q be any permutation pattern of length k. 
Then Avn(q) < (2k)2”. 


In 2013, Jacob Fox [178] disproved Conjecture 4.97. He proved the following 
result. 


THEOREM 4.98 
There exists an infinite sequence 7, 72,--- of patterns so that Tz is a pattern 


of length k, and if f(k) = L(ap%), then 
f(k) ” ger) 


In other words, cz is an exponential function of k, not a polynomial function 
as Conjecture 4.97 claimed. We note that Fox’s proof of Theorem 4.98 is 
nonconstructive, so we still do not have an example of an infinite sequence of 
patterns that exhibit the behavior described in the theorem. 

We mention that Jacob Fox has also improved Cibulka’s upper bound for 
cr that we stated in (4.16). See Problem Plus 28 for that result. 


Dr 


Exercises 


1. Prove that Av,,(123---&) = Av,(123---k(k —1)). Here the pattern on 
the right is the monotone pattern of length k, with its last two entries 
reversed. Do not use Theorem 4.23. 


2. (a) Find a formula for Av,41(q) where gq is any pattern of length k. 


(b) Find a formula for Avyz+2(q) where g is any pattern of length k. 


196 


Co ont nn 


10. 


11. 


12. 


13. 


14. 


15. 


Combinatorics of Permutations, Third Edition. 


. For patterns p and q, let Av,,(p,q) denote the number of permutations 


of length n that avoid both p and q. Find a formula for Av,,(132, 312). 


. Find a formula for Av,(p, q), where p and q are patterns of length three. 


Go through all possible choices of p and g. How many different sequences 
are there? 


. Find a formula for Av,,(132, 1234). 
. Prove that f(n) = Av, (132, 3421) = 1+ (n—1)2"-?. 
. Prove that g(n) = Av,(132, 4231) = 1+ (n—1)2"-?. 
. Find a formula for Av, (132, 4321). 


(a) Find the ordinary generating function >,..9) Avn (3142, 2413)z”. 


(b) Find other pairs of patterns (p, g) so that Av, (p,q) = Avn (3142, 2413). 


(Do not look for the easy way out. We are interested in pairs that 
cannot be obtained from (3142, 2413) by iterating the trivial equiv- 
alences.) 


The result of Exercise 6 shows that Av,(132,3421) is always odd if 
n > 3. Prove this fact by finding an involution on the set enumerated 
by Av,,(132, 3421) that has exactly one fixed point. 


(a) Let h(n) = Av,,(1324, 2413). Compute the ordinary generating 
function H(z) = >?) A(n)z”. 


(b) Find other pairs of patterns (p, g) so that Av, (p,q) = Avy (1324, 2413). 


Again, we are looking for nontrivial examples. 


In Exercises 6 and 7, we have seen examples for pairs of patterns p and 
q so that Av,,(p,q) is odd for sufficiently large n. Are there pairs of 
patterns p and q so that Av,,(p,q) is even for sufficiently large n? 


Let [,,(q) be the number of involutions of length n that avoid the pattern 
q. Find the ordinary generating function for the numbers I, (2143). Note 
that involutions enumerated by I,(2143) are called vezillary involutions. 


Are two patterns less restrictive than three patterns? More precisely, 
let a, b, c, d, and e be five patterns of the same length. Is it then true 
that 

Avn(a,b) > Avn(c, d,e) 


as long as (a,b) # (123---k, k(k —1)---1)? 


Find an explicit formula for the number of 132-avoiding n-permutations 
that are decomposable into three skew blocks, each block consisting of 
a skew indecomposable permutation. 
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16. 


17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


(a) Find a formula for the number of skew indecomposable 321-avoiding 
permutations. 


(b) Find a bijective argument proving the result of part (a). 


(—) Find a bijection f from the set of 231-avoiding n-permutations to the 
set of northeastern lattice paths from (0,0) to (n,n) that do not go above 
the main diagonal. What parameter of these paths will correspond to 
the number of ascents of the corresponding permutations? 


Give an example of two patterns p and q so that Av,,(p,q) is a polyno- 
mial. 


(—) Are there any patterns p and q so that Av,(p,q) = 0 if n is suffi- 
ciently large? 


Let E,(q) (resp. On(q)) denote the number of even (resp. odd) permu- 
tations of length n that avoid a given pattern g. Prove that 


Cr+ Cine 
20393 
2 
Prove that 
Ey,(231) — On(231) = E,(312) — On (312) = (-1)4"/ Caa_ty 2. 
Prove that 
E,(213) — On (213) = Con—1y/2- 

Give an example of three patterns m, g, and r so that 


lim ¥/Av,(m,q,r) 
nM co 
is not an integer. 


Let us assume that we know the numbers Av“) (q) of permutations of 
length n having exactly k fixed points that avoid a certain pattern q. 
For what other patterns q’ can we obtain the numbers Av) (q’) directly 
from these data? 


. Prove that if k > 3, then there exists a pattern of length k so that for n 


sufficiently large, the inequality Av,,(123---k) < Avn(q) holds. 


. Find a formula for [,,(132). 


(a) A binary plane tree is a rooted plane tree with unlabeled vertices 
in which each vertex that is not a leaf has one or two children, and 
each child is either a left child or a right child of its parent. So the 
two binary plane trees shown in Figure 4.14 are different. Prove 
(preferably by a bijection from the set of 132-avoiding permutations 
of length n to the set of binary plane trees on n vertices) that the 
number of binary plane trees on n vertices is Cy. 
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Two distinct binary plane trees. 


28. 


29. 


30. 
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32. 


(b) What parameter of these plane trees will correspond to the number 
of descents of the corresponding 132-avoiding permutation? 


(c) How can we decide from a binary plane tree whether the corre- 
sponding 132-avoiding permutation has a descent in i? 


Let D,,(k) denote the number of 321-avoiding n-permutations that start 
in an ascending run consisting of exactly k elements. Prove that 


2n—k-1 2n-—k-1 
D,(k) = -_ : 
ce. 
Prove by a simple direct argument that Av,,(1234) and Av,,(1324) are 
not asymptotically equal. 


Let r be a pattern that starts with the entry 1, and assume that Av,,(r) < 
c” for some positive constant c. Let w= 1@q. Prove that Av,,(w) < 
(c+1+2Vc)”. 


Let q be a pattern that has an entry x that is a left-to-right maxi- 
mum and a right-to-left minimum at the same time, and assume that 
Avn(q) < c” for all n. In other words, we have q = L 614 R for some 
strings Z and R. Let us replace x in q by a pattern p so that the ob- 
tained pattern q’ has form L@p@ R. Let us assume that Av,,(p) < C” 
for all n. 


State certain conditions for p, then prove that if those conditions hold, 
then there exists a constant K so that Av,(q’) < (4cC)” for all n. 


Prove that the number of 1324-avoiding n-permutations in which the 
entry 1 precedes the entry n is less than 8”. 
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33. 


34. 
35. 


36. 
37. 


38. 


39. 


40. 


41. 


42. 


43. 


Let q be a pattern of length k. Prove that Av,n(q) > c(k—1)”, for some 
positive constant c. 


For what positive integers n will Av,,(123) be an odd integer? 


The bijection F that we constructed in the proof of Theorem 4.59 
maps the set of indecomposable 1342-avoiding n-permutations to the set 
DEO of 8(0, 1)-trees on n vertices. Let A be the set of 123-avoiding 
n-permutations that are in the domain of F’. Describe F(A). 


Find a formula for [,,(231). 


Let f be the bijection defined in Exercise 17. Let B,, be the set of 
permutations in the domain of f that have exactly m inversions. Give 
a simple description of f(Bm). 


Find three patterns p, g, and r so that for all positive integers n, the 
equality Av, (p,q,7) = 7 holds. 


Are there infinitely many nontrivial pairs of patterns p and q so that 
Avn(p,q) < hip,q)(n) for all n, where h(p,q)(n) is a polynomial function 
of n? By nontrivial pairs we mean pairs in which at least one of the two 
patterns has at least two alternating runs. 


Find a formula for the number of 132-avoiding n-permutations whose 
longest decreasing subsequence is of length exactly k + 1. 


Find a formula for the number B,(a1, a2, a3) of all permutations of the 
multiset {1% 2723} that avoid both 123 and 132. 


Let us extend the notion of permutation pattern avoidance for words 


over a finite alphabet {1,2,---,m} as follows. We say that a word w 
contains the permutation pattern q = qiq2--- qa if we can find d distinct 
entries in w, denoted a1, a@2,--- , aq from left to right so that a; < a; if 


and only if dj < d;. 

We say that a word ¢t on the alphabet M = {1,2,--- ,m} is k-regular if 
the distance between two identical letters in ¢ is at least k. Let I,(q,m) 
be the maximum length that a k-regular word over M can have if it 
avoids the permutation pattern g. Let us assume that we know that 
there exists a constant ¢ = Ck,g,m so that 


In(q,m) < cm. (4.17) 
Prove the Stanley—Wilf conjecture from this assumption. 


Let 
110 
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Let the sequence of matrices A; be defined by Ap = 1, A, = € i): 


10 
Ign An . . . . . n n 
and Anyi = A oP where In is the identity matrix of size 2” x 2”. 
Set m = 2”, and prove that f(m, B) = Q(mlogm), so inequality (4.14) 
of the Fiiredi-Hajnal conjecture does not hold for B. 


Let us say that the permutation p = p,p2---py tightly contains the 
permutation gq = q1q2--: qe if there exists an index 0 < 1 < n—k so that 
qj < dr if and only if pi4; < pir. In other words, p tightly contains q 
if there is a string of k entries in p in consecutive positions that relate 
to each other as the entries of g do. 


If p does not tightly contain q, then we say that p tightly avoids q. Let 
Tn(q) denote the number of n-permutations that tightly avoid q. For 
instance, 1436725 tightly contains 123 (consider the third, fourth, and 
fifth entries), but tightly avoids 321 or 4231. 


Prove that if n > 4, then T,,(123) > T,,(132). 
Prove that if n > 7, then T,,(1342) > T,, (1243). 


Find a direct bijection between 321-avoiding permutations of length n 
and lattice paths using steps (0,1) and (1,0) from (0,0) to (n,n) that 
never go above the main diagonal. 


Alternating permutations are defined in Definition 1.54. Find a formula 
for the number of 312-avoiding alternating permutations of length n. 


Find all pairs of patterns (p, q) of length three or more so that Av,,(p, qg) = 
I,(p,q) for all n. That is, all permutations that avoid p and q are invo- 
lutions. 


Let g be a skew indecomposable involution. Prove that for all n, the 
inequality 
Avn(q) < Ion (q) 


holds. 


Use the framework of Theorem 4.40 to prove an upper bound for 


limsup V/ Ip (1324). 


Let k > 3, and let Hy be the set of k — 1 patterns of length k that 
start with an increasing subsequence of length k — 1, and end with a 
descent. For instance, H4 = {1243, 1342, 2341}. Prove that Av, (Hx) < 
((k — 1)? +1)”. 
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52. 


53. 


54. 


55. 


56. 


57. 


58. 


59. 


60. 
61. 


62. 


Let u,(q) denote the number of permutations that avoid gq and have a 
unique longest increasing subsequence, or ULIS. Find the closed form 
of the generating function >>.) Un(231)2”. 


A Dyck path of semilength n is a lattice path from (0,0) to (2n, 0) using 
steps (1,1) and (1,—1) that never goes below the horizontal axis. 


(a) Find a natural bijection f from the set Av,,(132) to the set of all 
Dyck paths of semilength n. 

(b) Let UI,,(132) be the set of all 132-avoiding involutions of length 
n that have a ULIS. Describe the image of UJ,,(132) under the 
bijection f of part (a). 

Let us say that permutation p strongly avoids pattern q if p and p? 

both avoid g. Let Sav,,(q) denote the number of strongly g-avoiding 
permutations of length n. Find an explicit formula for the ordinary 
generating function for the sequence of numbers Sav,,(312). 


Let k be a positive integer, and let n > (k — 1)? +1. Prove that 
Sav,(12---k) =0. 


Let q' be the reverse complement of g. Prove that for all n, the equality 
Savn(q) = Savn(q’) holds. 


(+) Let us call permutations of length n that consist of one n-cycle 
cyclic. Let Cyc,,(q1,q2) denote the number of cyclic permutations of 
length n that avoid the patterns q, and qz. Find an explicit formula for 
Cyc,, (123, 132). 


Prove the following formulas. 

(a) For n > 5, Cyc, (123, 321) = 0. 
(b) For n > 3, Cyc,,(231, 312) = 0. 
(c) 

(d) 


Let ¢ = q19¢2°°: dx be any involution of length k > 2 such that if q; =k, 
then i < k — 2. Prove that for all n > 2, the inequality 


2Cyc,(q) < Cyen44 (q) 


For all positive integers n, Cyc, (231,321) = 1. 
For all positive integers n, Cyc, (132,321) = ¢(n). 


holds. 
Prove that Avy (1234, 1243) < (3 + 2/2)”. 


Let & > 2. Find a permutation class C so that its number of elements 
of length n is equal to the number of compositions of the integer n into 
parts that are at most k. 


(-) Are there any real numbers in the interval (0,1) that are growth 
rates of a permutation class? 
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a 


Problems Plus 


1. Prove that the number of n-permutations that avoid 132 and have k+1 


left-to-right minima is +('') ea : 


2. Prove a formula for the number d,,(132) of derangements of length n 
that avoid 132. 


3. (a) Let p be a permutation that is the union of an increasing and a 
decreasing sequence. Such permutations are called skew-merged. 
Can we characterize p in terms of pattern avoidance? 


(b) Find a formula for the number of skew-merged n-permutations. 


4. (a) Prove that 

4qn-l + 2 
aa 

(b) Find other pairs of patterns p and q so that the equality Av, (p, gq) = 
Avy,(1234, 3214) holds for all n, besides those pairs obtained from 
(1234, 3214) by trivial symmetries. 


Avy (1234, 3214) = 


5. Find an upper bound for Av,,(23451, 13452, 12453). 


6. Prove that for all positive integers n, the equality Av,,(1243, 2143, 321) = 
n+ 2(%) holds. 


7. Prove that for all positive integers n, the equality Av, (1243, 2143, 231) = 
(n + 2)2”-3 holds. 


8. Let us generalize the notion of pattern avoidance as follows. A gener- 
alized pattern is a pattern in which certain consecutive elements may 
be required to be consecutive entries of a permutation. For instance, a 
generalized 31 — 2 pattern is a 312-pattern in which the entries playing 
the role of 3 and 1 must be consecutive entries of the permutation (this 
is why there is no dash between 3 and 1), but the entries playing the 
role of 1 and 2 do not have to be consecutive (this is why there is a dash 
between 1 and 2). 


For example, a 3 — 1 — 2 generalized pattern is just a traditional 312- 
pattern, a 2—1 generalized pattern is an inversion, while a 21 generalized 
pattern occurs when the permutation has a descent. 


Prove that Av,,(1—23) = B,,, where B,, is the nth Bell number, denoting 
the number of all partitions of the set [n]. 


9. Is generalized pattern avoidance always stricter than traditional pattern 
avoidance? That is, let q be a traditional pattern, and let q’ be a gener- 
alized pattern so that q’ becomes q if all the consecutiveness restrictions 
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11. 


12: 


13. 


14. 


15. 


16. 


17. 


are released, but there are some consecutiveness restrictions. Is it then 
true that Av,,(q) > Av,(q’) if n is large enough? 


For a permutation p, and a generalized pattern q, let q(p) denote the 
number of occurrences of g in p. So the examples of Problem Plus 8 can 
be written as 


d(p) = 21(p), and i(p) = (2—1)(p). 
Find a similar expression for maj(p). 


In Exercise 42, we proved the Stanley—Wilf conjecture from formula 
(4.17), which we assumed without proof. Prove formula (4.17). 


Let q be any pattern of length k — 3. Prove that for all n, the equality 
Avn((k — 1)(k — 2)kq) = Av, ((k — 2)k(k — 1)q) holds. 


Prove that for any permutation patterns q of length k, the inequality 


Paaee k? 
L(q) = lim VAva(a) 2 G 
holds. 


Let q be an indecomposable pattern, let L(q) be defined as in the pre- 
vious Problem Plus, and let g’ =14q. 


(a) Prove that L(q’) > 1+ L(q) + 2\/L(q). 
(b) Prove that in the special case when gq starts in the entry 1, the 
equality L(q’) =1+ L(q) + 2\/L(q) holds. 


Let f(z) = 0,59 AVn(1324, 4321)z”. Prove that 


1 — 12z + 592? — 15229 + 21824 — 16825 + 5826 — 627 
(1 — z)(1 — 2z)4(1 — 42 4 22?) 


Use this fact to compute D(1324, 4321). 


Let T,,(q) be defined as in Exercise 44. Prove that for any given pattern 
q, there exists a constant c, so that 


lim V/Tn(q)/n! = cq. 


n—->co 


Prove that for all positive integers k > 3, the identity 


yn gtk gtk+ = 
mB Wars |X cay” 2 EFT 


n>0 i>0 i>0 


holds. 
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Find a closed formula for A(z) = >, Tn(123)2,. 


A permutation pattern q = q1q2--- qx is called nonoverlapping if there 
is no permutation p = p,p2--:pn so that k < n < 2k —1 and both 
Pip2°+*Pk and Pn—k41Pn—k+2°°*Pn form a q-pattern. For instance, g = 
132 is nonoverlapping, but gq’ = 2143 is not since p = 214365 has the 
property that both its first four entries and its last four entries form a 
2143-pattern. In other words, a permutation is called non-overlapping 
if it is impossible for two of its copies to overlap in more than one entry. 
Let ay be the probability that a randomly selected n-permutation is 
nonoverlapping. 


(a) Prove that if n is even, then 


ss 
an =1— De (4.18) 


(b) What modification of (4.18) is necessary if n is odd? 


(c) Prove that lim,+. @, exists, and compute this limit up to four 
decimal points. 


Let gq be a nonoverlapping pattern of length k, and let V,,(q) be the 
number of permutations of length n that do not contain a very tight 
copy of g. A tight copy of q is called very tight if the set of entries of 
q forms an interval. For instance, in 25687314, the entries 5687 form a 
very tight copy of 1243, but the entries 314 do not form a very tight 
copy of 213. Prove that V,(q) < Vn(12---k). 


For a permutation pattern r, let r’ =r@1. 


Prove the following generalized version of Lemma 4.38. Let qi, gz, and 
q3 be three permutation patterns. Then every permutation avoiding 
q1 450 Q3 is a merge of a permutation avoiding qi @q), and a permutation 
avoiding gq ® q. 


Use the result of the preceding Problem Plus to prove Theorem 4.96 in 
the following stronger form. Let q be the layered pattern of layer lengths 
Q1,42,°°: , a. Then 


t-1 2 
Avn(q) < («1 +e-t41 4250) : 
4=2 


Reverse alternating permutations were introduced in Definition 1.54. 
Let RA,(q) denote the number of reverse alternating n-permutations 
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that avoid the pattern g. Prove that for all n > 1, the equality 


2+ (3n)! 


As (O43) = RAs (934) 
BARONESS) n(n + 1)\(n + 2)! 
holds. 


Keep the notation of the previous problem. Prove that for all n > 1, 
and for all k, the equality 


RAg,(12-++k) = RAgn(12-+-k(k — 1)) 
holds. Explain why the identity does not hold if 2n is replaced by 2n+1. 


Keep the notation of the previous problem. Prove that for all n (odd or 
even), the equality 


RA, (12---k) = RA,(21---(k — 1)k) 
holds. 
Prove that for all positive integers n, the equality 
Av,,(1324) = Av, (1234, 3416725) 
holds. 


Prove the following version of Theorem 4.98. For each k, there exists 
a family U consisting of almost all patterns of length k so that L(U) 


exists, and L(U) = 2((#/log kr)" Tn other words, even if we count 
permutations that simultaneously avoid almost all patterns of length k, 
we get the described, relatively high number. 


Recall the notation c;, = maxges, L(q). Prove that cj, = O (2*). 
Find the generating function for the sequence Av,,(3124, 4312). 


Prove that (132) 
Un 
lim ——— =1/2. 
im G / 


n—->co n 


(See Exercise 52 for the definition of up, (132).) 


Find an asymptotic formula for the number of 132-avoiding involutions 
of size n that have a unique longest increasing subsequence. 


Prove that i 
Cyc, (132, 231) = — dygr/4. 
ye, (182, 231) ra a, u(d) 


See Exercise 57 for the definition of Cyc, (q1, q2). 
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Let ¢ be the Euler totient function. That is, for a positive integer z, let 
¢(z) be the number of positive integers less than z that are relatively 
prime to z. Prove that 


(2k) = o(n/2) if n = 4k, 
Cyc,, (123, 231) = ¢ o(k +1) + (2k + 1) = 6(4) + ¢(8) ifn=4k4+2, 
o(m) = o((n + 1)/2) ifn = 2m —- 1. akg 
4.19 


Prove that for all positive integers n, the inequality Cyc,,(132, 213) < 
n?2”/? holds. 


Let C be a permutation class, and let |C,,| be the number of elements 
of C that are of length n. Prove that exactly one of the following two 
statements holds. 


(a) There is a constant K so that |C,| < n*, for all n > 1. 


(b) For all n > 1, the inequality F, < |C,,| holds, where F,, is the nth 
Fibonacci number, defined by Fo = F, = 1, and F, = F,-1+ Fn—2 
for n > 2. 


Provide an example when equality holds in the statement in (b). 


The preceding Problem Plus shows an example of a permutation class C 
whose growth rate (which is equal to (1+ V5)/2) is in the open interval 
(1,2). What other numbers in the interval (1,2) are growth rates of 
permutation classes? 


What is the smallest real number c > 2 so that c is the growth rate of 
a permutation class? 


What is the smallest real number « so that there are uncountably many 
permutation classes that have growth rate «? 


It follows from the answer to Problem Plus 36 that the subset of the 
interval (1,2) that consists of growth rates of permutation classes is 
countable. What is the largest real number € so that the same statement 
remains true for the interval (1, &)? 


Recall that a real number r is algebraic if there exists a polynomial A(z) 
with rational coefficients so that A(r) = 0. Is it true that the growth 
rate of a permutation class is always an algebraic number? 


Does there exist a real number \ so that if A < c, then c is the growth 
rate of a permutation class? 
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Dn 


Solutions to Problems Plus 


1. First, note that in a 132-avoiding permutation, the entry p; is a left- 
to-right minimum if and only if either 7 = 1, or 2 — 1 is a descent. 
Therefore, we are looking for the number of 132-avoiding n-permutations 
with & descents, or, by taking the reverse, 231-avoiding n-permutations 
with k ascents. We know from Exercise 17 that the number of such n- 
permutations is equal to the number of northeastern lattice paths from 
(0,0) to (n,n) that have k north-to-east turns and that never go above 
the main diagonal. A comprehensive survey of lattice paths of this and 
more general kinds can be found in [233]. 


In order to count these paths, note that the north-to-east turns of such 
a path completely determine the path. It therefore suffices to count the 
possible positions for the k-element set of north-to-east turns of such a 
path. Let (a;,b;) be the coordinates of the ith north-to-east turn of a 
northeastern lattice path r; then the vector 


Q1,42,°"° Gk, 01, ba, +> » OR (4.20) 


completely determines r. Disregard for the time being the requirement 
that r does not go above the main diagonal. Then we have 


O< a, <ag<-::<aycn-1 


and 
L<bi < bg <-++ <b <n. 


Clearly, all vectors (4.20) satisfying these conditions define a northeast- 
ern lattice path with k turns, proving that the number of these lattice 
paths is (7) (7). 

Now we have to count the vectors (4.20) defining a lattice path that 
goes above a main diagonal, that is, we have to determine for how many 
vectors (4.20) there exists an index 7 so that a; < b;. Let s be such a 
vector, and let 2 be the largest index for which a; < b;. Define 


f(s) = (bi, ba, ++: bi-1, Gita, *** »4k,Q1,°°° G4, Dy, ++ , bx). 
Then f(s) satisfies the chains of inequalities 


1< by < ba S++ < by s+ Say S++ Sap n—1 


and 
O< a, Sag <-++ Saye Sig Se SOR SS — 1. 


Therefore, we have (71})(771) possibilities for f(s). As f is a bijec- 
tion (this has to be shown), this means that this is also the number of 


ca 
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possibilities for s. So the number of northeastern lattice paths with k 
north-to-east turns that do not go above the main diagonal is 


n\ (n m+1\/n—-1\_ 1/n n 

kk} \k k+1/\k-1/° n\k/ \k41/° 
These numbers are often called the Narayana numbers, and are denoted 
A(n, k). 


2. It is proved in [276] that 


1 n—2 =4 a 


i=0 


The numbers d, are called the Fine numbers. An alternative formula 
for these numbers is 


4,(132)= Gna ‘ 7 ine 


1<k<n/2 


3. (a) It is proved in [288] that the skew-merged permutations are pre- 
cisely the permutations that avoid both 2148 and 3412. 


(b) Michael Atkinson [19] proved that the number of these permuta- 
tions is 


n—-1 
Av (2143, 3412) = (*") > pooa(2m), 
a m 


m=0 


4. (a) This result was proved in [235] by Darla Kremer. 
(b) It is proved in [235] that 


4-149 
— = Av,,(4123, 3214) = Av,,(2341, 2143) 


= Av,,(1234, 2143). 


5. The number of these permutations is less than (6 + 4/2)". Take a 

permutation p € of length n that is in that class. Remove its left-to- 
right minima. We get a permutation p’ that must avoid 1342, otherwise 
p would have contained 12453. More interestingly, p’ must also avoid 
2341, because otherwise p would have contained either 13452 or 23451, 
and both are impossible. 
So p’ avoids both 1342 and 2341. Such permutations are counted by the 
Schréder numbers, which have exponential order (3 +2V2) = (1+ V2)?. 
We mentioned these numbers in the solution of Exercise 9. The proof 
is then analogous to that of Lemma 4.37. 
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6. 


10. 


11. 


12. 


This result is due to Astrid Reifegerste [274], who explored the connec- 
tions between permutations counted by Schroder numbers and lattice 
paths. 


This result is due to A. Reifegerste, ibid. 


The concept of generalized patterns was introduced in [25]. This result 
was proved in [120] in two different ways, together with many other re- 
sults concerning short generalized patterns. One way is to recall that the 
Bell numbers satisfy the recurrence relation B(n + 1) = )>}_-) B(i)("), 
and then show that the 1-23 avoiding permutations satisfy this same 
recurrence. The other way is a direct bijective proof, based on a way 
of writing each partition in a canonical form similar to what we have 
seen for permutations in Chapter 3. (Write each block with its small- 
est element first, and then in decreasing order, then order the blocks in 
decreasing order of their smallest elements.) 


No, that is not true, and one can find a counterexample using any of the 
four non-monotone patterns of length three. For instance, a permutation 
contains 132 if and only if it contains 13-2. The “if” part is obvious. 
For the “only if” part, let acb be a 132-pattern in p, and assume that 
the distance between the positions of a and c is minimal among all 132- 
patterns in p. That implies that if there is an entry d located between 
a and c, then d cannot be less than a (for dcb), cannot be larger than b 
(for adb), and cannot be between a and 0 in size (for dcb). This means 
there cannot be any entries among a and c. 


This result is from the paper that introduced generalized patterns [25]. 
In that paper, the authors showed that essentially all Mahonian statistics 
in the literature can be expressed by generalized patterns. For the major 
index, one just has to add the descents of p; in other words, count the 
entries that precede a descent, then add these numbers for all descents. 
With this in mind, it is straightforward to see that 


maj(p) = (1 — 32)(p) + (2 — 31)(p) + (3 — 21)() + (21)(p). 


This result can be found in Martin Klazar, The Fiiredi-Hajnal con- 
jecture implies the Stanley—Wilf conjecture, Formal Power Series and 
Algebraic Combinatorics, Springer, Berlin, 250-255, 2000. In that pa- 
per, the author in fact proves that formula (42) and the Fiiredi-Hajnal 
conjecture are equivalent. 


See [289], where the authors prove this result introducing the interesting 
notion of shape—-Wilf-equivalence. For two patterns g and q’, it is nec- 
essary for Av,(q) = Avn(q’) to hold for all n in order for the patterns 
to be shape-Wilf-equivalent, but it is not sufficient. 
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This result was proved by Pavel Valtr, but was first published in [223]. 
One can assume without loss of generality that gq is indecomposable, and 
then build a sufficient number of decomposable permutations in which 
each block avoids q. 


(a) See [57] for a proof. 


(b) This follows from part (a), and the proof of part (b) of Exercise 
30. 


This result is due to Michael Albert, Michael Atkinson, and Vincent 
Vatter [6]. As f is a rational function, the exponential order of its 
coefficients is equal to the reciprocal of its singularity of the smallest 
modulus, which is 2+ V2. 


This result is due to Sergi Elizalde [159]. It is worth pointing out that 
Elizalde and Marc Noy [158] conjectured that if q is any pattern of 
length k, then T,,(q) < T,(12---k&). Recently, in [160], Elizalde proved 
this conjecture. Furthermore, in the same paper, he proved that if q is 
any pattern of length k, then T;,(q) > 12---(k — 2)k(k — 1). 


This result is due to Richard Warlimont [323, 324]. 


It has been proved by Sergi Elizalde and Marc Noy [158] that 


e2/2 
A(z) = vs, 


2 cos (B24 z) 


In that same paper the authors provide a (somewhat less closed) formula 
for the corresponding generating function for the tight pattern 132. 


(a) If g is nonoverlapping, then there is a smallest index j so that the 
first and last j entries of g form identical patterns. That pattern 
has to be non-overlapping. 


(b) If n = 2k +1 is odd, then there are overlapping patterns of length 
n for which the smallest index 7 described in the solution of part 
(a) is 7 =k+1. If 6, is the number of such patterns, then a, = 


n/2 aj 
LS aor ew 
(c) It can be shown that bo,41 < ag41/(k + 1)!, and so the sequence 


of the a, is monotone decreasing. The details can be found in [68]. 
The limit L is close to 0.3641. 


The proof of this result can be found in [61]. The main idea of the proof 
is that if g is nonoverlapping, then it is very difficult to pack many very 
tight copies of q into a permutation, so many permutations are needed 
in order to contain all very tight copies of q. 
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21. 


22. 


23. 


24. 


25. 


26. 


27. 
28. 
29. 


30. 


This result has been published in [119]. The proof is similar to that of 
Lemma 4.38. 


This result has also been published in [119]. Let q be the layered pattern 
of layer lengths aj, a2,--- ,a:. The claim of Theorem 4.96 is proved by 
induction on t, the case of t = 1 being obvious, and the case of t = 2 
directly following from Theorem 4.23. Let us assume that we know the 
statement for all positive integers less than t. We can now use Lemma 
4.37 with q, being the layered pattern consisting of two layers, of length 
a, and ag, and q2 being the layered pattern consisting of t — 1 layers of 
length a2, a3,--- , a, to get the desired upper bound. 


This result is due to Joel Brewster Lewis [243], who proved that both 
sides are equal to the number of Standard Young Tableaux (which will 
be defined in Chapter 7) of shape (n,n, 1). 


This result is due to the present author [70], who proved that if the 
length of our permutations is even, then the bijection of Exercise 1 
preserves the alternating property. 


The present author proved [70] that a slight modification of the bijection 
used in the solution of the previous problem is possible here, and parity 
restrictions are not needed. 


See the paper of Alexander Burstein and Jay Pantone [104] that contains 
the first examples of this kind of unbalanced Wilf equivalence. 


See [178] for this surprising result. 
This result is also proved in [178]. 
Jay Pantone [259] proved the exact formula 


(82° — 16z* + 2823 — 2627 + 9z —1) 4 JT — 4e(22* — 823 4 142? — 72 41) 
22?(1 — 6z + 92? — 423) 


for this generating function. Note that the denominator has rational 
roots. 


The number u,,(132) is equal to the number of plane unlabeled trees on 
n+ 1 vertices that have a unique leaf at a maximum distance from the 
root. Then our statement is the special case of the following general 
theorem. Let us select a rooted plane unlabeled tree on n vertices uni- 
formly at random, and let a,x be the probability that the selected tree 
has & leaves at maximum distance from the root. Then 


lim Gn.b = a 
noo 
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A very general theorem that contains this result is Theorem 2 in [263]. 
An earlier, and more specific, reference is [213]. 


A more direct, more elementary proof would certainly be welcome. 


It follows [81] from the solution of Exercise 53 that such involutions are 
in bijection with lattice paths of length n+1 consisting of U steps and D 
steps so that each prefix and each suffix contains strictly more U steps 
than D steps. The main term of their asymptotic enumeration can be 
found in [90], while the most precise enumerative result is proved in 
[208], where these lattice paths are called bidirectional ballot sequences, 
and the number of such paths of length n is denoted by B,,. The result 
is that 


In particular, 
gn+l gn— 1 


Mee =f 
OE) eS ay adel 


This was the first result of this kind, and it can be found in [15]. 
This result is proved in [80] using a case-by-case analysis. 
This result is due to Brice Huang [218]. 


This result was proved by Tomds Kaiser and Martin Klazar in [223]. 
The class Av,,(123, 132,213) = F,, is an example for equality. 


Let the generalized Fibonacci numbers F;,, be defined by the identity 


mr 1 
> Fake =~ Ja 27 — 272 — 2k 


n>0 


Then it is proved in [223] that exactly one of the following two state- 
ments hold for any infinite class C of permutations. 


(a) There is a unique & and a constant c so that Fix < |Cn| < Fri: n° 
for alln > 1. 


(b) The inequality |C,| > 2”~' holds for all n > 1. 


It follows that the only numbers in (1,2) that are growth rates are those 
that are growth rates of the generalized Fibonacci numbers F;,,, for 
some fixed k. These are the reciprocals of roots of the smallest modulus 
of the polynomials (1 — z — z? —--- — z*), or, equivalently, the roots of 
the largest modulus of (z* — z*~1! —..-—1), or, after multiplication by 
(z—1), of the polynomial z*+! — 2z* +1. In other words, there are only 
countably many growth rates in the interval (1, 2). 
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Vincent Vatter [316] has proved that the smallest such number c is the 
unique positive root of the polynomial 1 + 2z + z? + 23 — z4, which is 
about 2.06599. 


It is proved in [316] that the largest such value for « is the unique real 
root of the polynomial z? — 2z? — 1, so K & 2.20577. 


The smallest such real number is the unique positive real root of the 
polynomial 2° — 224 — 2? — z—1, so € & 2.30522. See [318] for a proof 
of this, and [260] for a characterization of all growth rates below €. 


No. In [9], Michael Albert and Steve Linton give an example of a per- 
fect set of growth rates. A perfect set is a closed set S so that each 
element s € S is an accumulation point of S\s. As any perfect set is un- 
countable, while the set of algebraic numbers is countable, non-algebraic 
growth rates exist. Note that all the known examples of non-algebraic 
growth rates involve permutation classes with an infinite base. Even 
more strongly, every known example of a growth rate of a finitely based 
permutation class is an algebraic integer (the root of a polynomial with 
rational coefficients and leading coefficient 1). 


Yes. The existence of such a real number A was first proved by Vincent 
Vatter [317], who also showed that the smallest possible value of 2 is 
less than 2.49. This bound was later improved to 2.36 by David Bevan 
[37]. 


Taylor & Francis 
Taylor & Francis Group 


http://taylorandfrancis.com 
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5.1 Polynomial Recurrences 
5.1.1 Polynomially Recursive Functions 


In the previous chapter, we spent considerable time and effort to find out how 
large the numbers Av,,(q) are. In this chapter, we will mostly concentrate on 
how nice they are, or rather, how nice the sequence {Avn(q)}i<n is. By abuse 
of language, we will often refer to this sequence as the sequence Av,,(q). 

First, of course, we should define what we mean by “nice.” We have already 
made one important definition, that of P-recursive (or polynomially recursive) 
sequences in Exercise 29 of Chapter 1, but for easy reference we repeat that 
definition here. 


DEFINITION 5.1 A sequence f : N > C is called P-recursive if there 
exist polynomials Po, Pi,--- , Py € Cin], with P,y 40 so that 


Py(nt+k)f(n +k) + Pei(nt+k—-1)f(n+k—1)4+---+ Po(n)f(n) =0 (5.1) 
for all natural numbers n. 


This definition, and some of the most important theorems in the theory 
of P-recursive sequences, can be found in [292], which is the earliest paper 
completely devoted to this subject. Chapter 6 of [297] is a comprehensive 
source for results on P-recursiveness. We do not want to duplicate existing 
literature on the topic, so we will not prove theorems that belong to the general 
theory of P-recursiveness. There will be one exception to this, Theorem 5.8. 

See the exercises following Exercise 29 of Chapter 1 for some examples of 
P-recursive sequences. 


Example 5.2 
The sequence of Catalan numbers is P-recursive. Therefore, the sequence 
Avn(q) is P-recursive for all patterns q of length 3. 
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PROOF We have 
Crh _ ie) n (2Qn)i(n—1)!(n—-1)ln — 4n—2. 


n 


GA wei Cc) — (Qn—2)nini(n+1) — n41’ 


therefore, (n + 1)C,, — (4n — 2)C,_1 = 0, providing a polynomial recurrence 
relation. | 


At this point, the reader should try to prove that the sum of two P-recursive 
sequences is also P-recursive. 

Example 5.2, and some other examples of P-recursive sequences Av,,(q), 
such as Av,,(1234), or in general, Av,,(123---), suggested the following con- 
jecture. 


CONJECTURE 5.3 [191] [Gessel, 1990] For any permutation pattern 
q, the sequence Avn(q) is P-recursive. 


Six years later, John Noonan and Doron Zeilberger looked at a larger family 
of sequences. Instead of trying to count permutations that avoid a pattern, 
they wanted to count permutations that contain it exactly r times, where r 
was a fixed non-negative integer. Numerical data and some theorems that we 
will cover in Chapter 7 led them to the following conjecture. 


CONJECTURE 5.4 [256] [Noonan and Zeilberger, 1996] Let Sy+(q) 
be the number of n-permutations that contain the pattern q exactly r times. 
Then for any fixed r and q, the function Sy+(q) is polynomially recursive. 


5.1.2 Permutation classes again 


It seems that Conjecture 5.4 is much more ambitious than Conjecture 5.3. 
Indeed, the latter seems to be a special case of the former, namely the special 
case in which r = 0. However, certain versions of the two conjectures are 
actually equivalent as has been proved by Atkinson [20]. 

Recall the definition of permutation classes that we made in Section 4.1.1. 
It is straightforward to check that the intersection of permutation classes, as 
well as the union of permutation classes, is always a permutation class. So, 
for instance, the class Av(p1,p2) of permutations avoiding both p; and po is 
a permutation class. 

Recall that in a partially ordered set, an element x is called a minimum if 
it is smaller than all other elements of the poset, while the element y is called 
minimal if no element of the poset is smaller than y. So a poset can have any 
number of minimal elements, but only one or zero minimum elements. 

Just as any ideal in any poset, a permutation class C is determined by the 
minimal elements of the complement of C’. These minimal elements obviously 
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form an antichain, that is, a set in which no two elements are comparable. 
This antichain is called the basis of C. 


Example 5.5 
The basis of the permutation class C'(p) is the one-element antichain consisting 
of p. 


Let C be a class that has basis p),p2,--- , px. The reader is invited to prove 
the simple fact that C = C(p1,p2,--- , px) must hold, that is, C must be the 
class of all finite permutations avoiding all p;, for 7 € [k]. 

Taking a second look at Conjecture 5.3, we see that it in fact claims that 
if f(m) is the number of n-permutations of a class C that has a one-element 
basis, then f(n) must be a P-recursive function. A stronger version of this 
conjecture was the following. 


CONJECTURE 5.6 For any permutation patterns qi, q2,°:- ,qk, the 
sequence Avn(q1,42,:°', Qk) is P-recursive. In other words, if f(n) counts 
the elements of a finitely based class of permutations that have length n, then 
f(n) is P-recursive. 


In other words, we replaced the condition of having a one-element basis by 
the condition of having a finite basis. 

Conjecture 5.6 was disproved in 2015 by Scott Garrabrant and Igor Pak 
[187]. They proved that there exists a set S of patterns of length 80 so that 
the sequence Av,,(S) of numbers counting permutations of length n that avoid 
all elements of S is not P-recursive. 


5.1.3 Algebraic and Rational Power Series 


Sometimes the generating function of a sequence has a much simpler form 
than the sequence itself. The following notion of d-finiteness, and the theorem 
after that, show how we can use these simple generating functions to prove 
P-recursiveness. 


DEFINITION 5.7 We say that the power series u(z) € C[[z]] ts d-finite 
if there exists a positive integer d and polynomials po(n), pi(n),--- ,pa(n) so 
that pa #0 and 


pa(z)u (z) + pa-1(z)ul?-(z) +++++p1(z)u'(z) + po(z)u(z) =0, (5.2) 
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We promised that we will prove one theorem from the theory of P-recursive 
functions. We are going to fulfill that promise now. The theorem in its explicit 
form is due to Richard Stanley [292]. 


THEOREM 5.8 
The sequence f(0), f(1),--- is P-recursive if and only if its ordinary gener- 


ating function 
= Ss (nz (5.3) 
n=0 


is d-finite. 


PROOF 


e First suppose u is d-finite, then (5.2) holds with pg # 0. Fix i < d. We 
start by finding an expression for p;(z) involving linear combinations 
of the f(n +t) with polynomial coefficients. Differentiate both sides of 
(5.3) i times and then multiply both sides by z/ to get 


Ju) =S"(nt+i-jif(nti-j)z”. (5.4) 


n>0 


Here (m); = m(m—1)-+-(m—i+1). Now let a; be the coefficient of z/ 
in p;(z). Multiply both sides by a;, then repeat the entire procedure for 
each nonzero coefficient of p;(z), and add the obtained equations. We 


get that 
pte = > (YF tn) 2% 


n>0 t 


where the q:(n) are polynomials in n, and the sum in the parentheses is 
finite. 


Repeating the above procedure for all « < d, and then adding all the 
obtained equations, we get an equation that is similar to the last one, 
except that on the left-hand side, we will have ae pi(z)u, which 
is, by (5.2), equal to zero. If we compute the coefficient of z"+* on 
both sides of the last equation and equate the two expressions, we get 
an equation that involves only linear combinations of some f(n + t), 
0<t<k, with coefficients that are polynomials in n. Therefore, this 
equation can be rearranged to yield a polynomial recurrence for f. This 
recurrence will not be 0=0 as pg # 0. 


e Now suppose g(n) is P-recursive in n, so (5.1) holds. Note that for any 
fixed natural number i, the polynomials (n+ 7);, 7 > 0, form a C-basis 
for the vector space C[n]. In particular, P;(n) is a linear combination of 
polynomials of the form (n+%);. Therefore, using generating functions, 
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Vanco Piln) f(n+i)z” is a linear combination of generating functions of 
the form 


Sl (n+ aj f(n +i) (5.5) 


n>0 
with complex coefficients. 
Compare formulae (5.4) and (5.5). We see that the left-hand side of 
(5.5) almost agrees with zJ~*u\), that is, they can only differ in finitely 
many terms with all negative coefficients. Let the sum of these terms 
be Ri(z) € z~'K[z7+], a Laurent-polynomial. If we multiply (5.1) by 
z” and sum over all non-negative n, we get 


0= (x: oye) + R(z). (5.6) 


Here the sum is finite by the definition of P-recursiveness and R(z) 
is a Laurent-polynomial. If we multiply both sides by z?% where q is 
sufficiently large, the terms with negative exponents will disappear and 
we get an equation of the form (5.2). 


LEMMA 5.9 
The product of two d-finite power series is d-finite. 


PROOF See [297], page 192. Il 


A useful consequence of Theorem 5.8 and Lemma 5.9 is that the convolution 
of two P-recursive sequences is P-recursive. 


LEMMA 5.10 
Let {f(k)}, and {g(m)}m be two polynomially recursive sequences, and let 


h(n) = $0 f(k)g(n — k). (5.7) 
k=0 
Then {h(n)}n ts a P-recursive sequence in n. 


PROOF _ Let F(z), G(z), and H(z) denote the ordinary generating func- 
tions of the sequences { f(k)}x, {g(m)}m, and {h(n)},. Then F(z) and G(z) 
are both d-finite by Theorem 5.8. It is well-known that F(z)G(z) = H(z), so 
H(z) is d-finite by Lemma 5.9; therefore {h(n)},, is P-recursive. 
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Note that by repeatedly applying Lemma 5.10, we get the statement that 
the convolution of several P-recursive sequences is always P-recursive in the 
sum of the variables. 

Knowing that a function is P-recursive is helpful, but often not sufficient 
to determine the function itself by interpolation. This is because we often do 
not know the degree of the recursion, or the degrees of the polynomials that 
appear in the recursion. Therefore, it is certainly useful to look for other, 
stronger properties that the sequence Av,,(q) has, at least for some gq. 


DEFINITION 5.11 The formal power series f € C[[z]] is called alge- 


braic if there exist polynomials Po(z), Pi(z),--- , Pa(z) € C[z] that are not all 
equal to zero so that 
Po(z) + Pr(z) f(z) +++» + Pa(z)f%(z) = 0. (5.8) 


The smallest positive d for which such polynomials exist is called the degree 


of f. 


Example 5.12 

Let f(z) = Invin a the generating function of the Catalan numbers. Rear- 
ranging the previous equation as 1 — 2zf(z) = 1 — 4z, then taking squares 
and rearranging again, we see that 


2” f(z) — zf(z)+2=0, 
so f is algebraic of degree 2. 


The following lemma shows the connection between algebraic and d-finite 
generating functions. 


LEMMA 5.13 
If f © Cl[z]] ts algebraic, then it is d-finite. 


PROOF See [297], page 190. Il 


Note that the converse of Lemma 5.13 is not true. The reader should try 
to find an example of a power series that is d-finite but not algebraic, then 
check Exercise 9. 

Even if having an algebraic generating function is a stronger property than 
being P-recursive, sometimes it is easier to prove the latter by proving the 
former. 


Example 5.14 
The sequence Av,,(1342) is P-recursive. 
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PROOF We have seen in (4.8) that the ordinary generating function 
H(z) of this sequence satisfies 


32z 


Ol ——— 
(?) = 34021 -( ~82 


We claim that H(z) is algebraic. To see this, it suffices to show that Z(z) = 
WG) is algebraic, because we can multiply both sides of the polynomial equa- 
tion satisfied by Z(z) = He by a power of H(z), and obtain a polynomial 
equation satisfied by H(z). On the other hand, routine transformations show 


that 


S41 


(322Z(z) + 82? — 202-1)” = (1-82), 


so Z(z) does satisfy a polynomial equation. | 


We would like to point out that the ordinary generating function of the 
sequence Av,,(1234) is not algebraic (see Problem Plus 2). So yet again, 
patterns of length four turn out to be quite surprising. The monotone pattern 
1234 is not only not the easiest or the hardest to avoid, it is also not the 
“nicest”! 

It is known that Av,,(123---k) is P-recursive for any k. We will prove this 
result in Chapter 7, when we learn about the interesting connections between 
permutations and Standard Young Tableauz. 

An even smaller class of power series is that of rational functions. A rational 
function is just the ratio of two polynomials, such as we. A rational 
function is always algebraic of degree 1 (as multiplying it by its denominator 
we get a polynomial), and therefore, d-finite. 


5.1.4 The Generating Function Of Most Principal Classes Is 
Nonrational 


DEFINITION 5.15 Let F andG be two combinatorial generating func- 
tions with nonnegative real coefficients that are analytic at 0, and let us assume 


that G(0) = 0. Then the relation 


1 


N=) 


(5.9) 
is called supercritical if G(Rg) > 1, where Re is the radius of convergence of 
G. 


The interested reader should consult [169] for a detailed treatment of su- 
percritical relations. 

Note that in the above definition, we allow for G(Rg) = oo > 1, which 
indeed happens when Re is a pole of G. 
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COROLLARY 5.16 

If the relation between F and G described in Definition 5.15 is supercriti- 
cal, then the radius of convergence of F is less than that of G, and so the 
exponential growth rate of the coefficients of F is larger than that of G. 


PROOF Note that as the coefficients of G(z) are nonnegative, G(R¢) > 1 
implies that G(a) = 1 for some a € (0, Re). So F(z) has a singular point 
closer to 0 than G(z) does. 


Example 5.17 
If F and G are two rational functions with nonnegative coefficients so that 
G(0) = 0 and (5.9) holds, then the relation (5.9) is supercritical. 


SOLUTION _ Let Re be the radius of convergence of G. It then follows 
from Pringsheim’s theorem that the positive real number Rg is a singular 
point of G. As G is a rational function, that singular point Re is necessarily 
a pole, so lim,_,r, G(z) = 00 > 1. i 


What does this mean for permutation classes? Let us recall that a permu- 
tation p is skew indecomposable if it is not possible to cut p into two parts 
so that each entry before the cut is larger than each entry after the cut. For 
instance, p = 2413 is skew indecomposable, but r = 364512 is not as we can 
cut it into two parts by cutting between entries 5 and 1, to obtain 364512. 

If p is not skew indecomposable, then there is a unique way to cut p into 
nonempty skew indecomposable strings $1, S2,--- , 8¢ of consecutive entries so 
that each entry of s; is larger than each entry of s; if 7 < 7. We call these 
strings s; the skew blocks of p. For instance, p = 67|435|2|1 has four skew 
blocks, while skew indecomposable permutations have one skew block. 

This leads to the following fact. Let A,(z) = 3°,3,Avn(q)z”, and let 
Aig(z) = ops AVn1(q)z” be the ordinary generating function of the se- 
quence of the numbers Av,,1(q) of skew indecomposable permutations of length 
n that avoid q. 


PROPOSITION 5.18 
Let q be a skew indecomposable pattern. Then the equality 
1 


A,(z) = Te Age 


(5.10) 


holds. 


PROOF The right-hand side is equal to 7,5, (A1,q(z))’, and in that 
sum, the summand indexed by i is the generating function for the number of 
g-avoiding permutations that have 7 skew blocks. 
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We will now show that for the generating function of most principal per- 
mutation classes, relation (5.10) is not supercritical, and therefore, by Propo- 
sition 5.18, these generating functions cannot be rational. Let L(g,1) = 
limp-sca (Avni(q))'””. It is easy to prove that L(q,1) exists and is finite, 
just as we proved that L(q) exists and is finite, in Section 4.3. 


LEMMA 5.19 

Let q = 9142°:: dk be a skew indecomposable pattern so that at least one of 
the equalities qi = 1 and qx, = k does not hold. Then the exponential growth 
rates of the sequences Av,(q) and Avni(q) are equal, that is, L(q) = L(q,1). 


PROOF Let us assume that qx #4 k. Let p € Aun. Let p)’ = p@1; in 
other words, p’ is p, with the new maximal entry n+ 1 affixed to the end. 
As qx # k, it follows that p’ avoids q, and as p’ ends in n +1, it is skew 
indecomposable. This injective map proves that 


Avn(q) < Avn41,1(9); 


0) 
L(q) < L(q, 1). 


As the inequality L(q,1) < L(q) trivially holds, our claim is proved. 

If qe = k, but q #4 1, then we can repeat the above argument for the 
reverse complement of q, which will not end in k, and which will be a skew 
indecomposable pattern. 


Now the proof of our main result concerning the nonrationality of the gen- 
erating functions of principal permutation classes is immediate. 


THEOREM 5.20 
Let q = 9142°:: dk be a skew indecomposable pattern so that at least one of 
the equalities qy = 1 and q, =k does not hold. Then the generating function 
A(z) is not rational. 


PROOF If A,(z) were rational, then by Example 5.17, the relation 
A,(z) = 1/(1— A1,q(z) would be supercritical, implying that L(q) > L(q, 1), 
but we saw in Lemma 5.19 that L(q) = L(q, 1). | 


Note that if q7 = 1 and q, = k, but q is Wilf-equivalent to a pattern q/’ 
that does satisfy the criteria of Theorem 5.20, then of course, Aq(z) = Aj,(z), 
so A,(z) is not rational. This happens for instance when q = 12---k, and 
q’ = 213-++k. 
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Note that Theorem 5.20 obviously fails for the pattern g = 12. Indeed, in 
that case, we have Av,,(q) = 1 for all n, so A,(z) = 1/(1— z). The shortest 
pattern for which we cannot decide whether A,(z) is rational is q = 1324. 

The following corollary will be useful in the Problems Plus. 


COROLLARY 5.21 

Let q be a pattern that is either equal or Wilf-equivalent to a pattern for 
which Theorem 5.20 applies. Let R be the radius of convergence of A,(z). 
Then Ag(z) < ov. 


PROOF — Lemma 5.19 shows that the convergence radius of A1,,(z) is also 
R. As the relation between A;,,(z) and A,(z) is not supercritical, it follows 
that Aj4(R) < 1, and so 


1 


= Ay gh) < Oo. 


A,(R) 


5.1.5 Polynomial Recursiveness of S,, ,(132) 


In this subsection, we present a fairly general result by proving the following 
theorem. 


THEOREM 5.22 

Let r be any fixed non-negative integer. Then the sequence Sy,,(132) is P- 
recursive. Moreover, the ordinary generating function of this sequence is al- 
gebraic of degree 2. 


The proof of this theorem will have a quite complicated, but elegant struc- 
ture. So that we do not get lost in the details, it is important to get an 
overview of our goals first. 

Note that no matter how large the number r is, it is a fired number. On 
the other hand, n is changing, and will eventually be much larger than r. So 
the requirement that our permutations have only r copies of 132 really means 
that they have these copies in exceptional cases only. 

In particular, every time there is an entry on the left of the maximal entry 
n that is smaller than an entry on the right of n, a 132-pattern is formed. 
This simple observation is crucial for us. Therefore, we introduce special 
terminology to discuss it. 

Entries of an n-permutation p on the left of the entry n will be called front 
entries, whereas those on the right of n will be called back entries. Front 
entries of p that are smaller than the largest back entry of p will be called 
black entries, whereas back entries of p that are larger than the smallest front 
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FIGURE 5.1 
A generic permutation and its colored entries. 


entry of p will be called gray entries. Note that not all entries will have colors. 
See Figure 5.1 for an illustration. 

Why are we coloring our entries? First, any black entry is smaller than any 
front entry which is not black, while any gray entry is larger than any back 
entry which is not gray. In other words, black entries are the smallest front 
entries, while gray entries are the largest back entries. Moreover, any black 
and any gray entry is part of at least one 132-subsequence. Indeed, take any 
black entry x, the entry n, and any back entry larger than x. A dual argument 
applies for gray entries. Finally, if a 132-subsequence spans over the entry n, 
that is, it starts with a front entry and ends with a back entry, then it must 
start with a black one and end with a gray one. 

Starting now, we are going to partition, and partition, and partition. That 
is, we will partition the set of n-permutations that contain r copies of 132 
into many equivalence classes, which we will call blocks, in order to avoid 
confusion with the notion of permutation classes. How many blocks? A lot, 
but the number of these blocks will not depend on n, just on r, so it will 
be a fixed number. Then we are going to show that within each block C, the 
number of permutations Sj32,-,¢(n) that are in C and have r copies of 132 isa 
polynomially recursive function of n. With that, we will be done by summing 
over all blocks C. Indeed, the sum of a fixed number of P-recursive functions 
is P-recursive. 

Let us start this partitioning. 


(a) As we said above, any colored entry is part of at least one subsequence 
of type 132 that spans over n. Therefore if p has exactly r subsequences 
of type 132, and B (resp. G) denotes the number of black (resp. gray) 
entries, then max(B,G) < r. This implies that we have at most r? 
choices for the values of B and G. 
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(b) 


Combinatorics of Permutations, Third Edition 


Once the values of B and G are given and we know in which position 
the entry n is, then we only have a bounded number of choices for the 
set of black and gray entries. Indeed, if x is the smallest black entry, 
then x is larger than all but G back entries. Thus, if n is in the ith 
position, then x > n—i—G. On the other hand, z is the smallest front 
entry, thus « <n—i+41. A similar argument applies for the largest 
gray entry. 


Finally, there is only a bounded number of positions where a black entry 
can be. Indeed, if x is black and y > « is a back entry, then 7 zy is a 
132-pattern for any front entry z on the right of x which is not black 
(and thus, is larger than any back entry). Recall that x ny is such a 
pattern as well. Thus, if ¢ is the number of such (a, z) pairs, then we 
have t+G <r. In particular, the distance between any black entry and 
the entry n cannot be larger than r. 


The following definition makes use of the observations we have just made: 


DEFINITION 5.23 We say that the n-permutations p, and p2 are in 
the same strong equivalence class if they agree in all of the following: 


position of the entry n 

set of black entries 

set of gray entries 

pattern formed by the gray entries 
position of the black entries 


pattern of consecutive entries starting with the leftmost black entry and 
ending with the entry n. 


In other words, permutations of the same block agree in everything that 
can be part of a 132-pattern spanning through the entry n. Note that once 
the position of n is given, there is only a bounded number of possibilities for 
the blocks of those permutations. 


DEFINITION 5.24 _ Let p be an n-permutation. The subsequence of p 
consisting of 


all black and gray entries and 
all front entries that are preceded by at least one black entry and 


the entry n 


is called the fundamental subsequence of p. 


In This Way, but Nicely. Pattern Avoidance. Follow-Up. 227 


This means that permutations of the same block have identical fundamental 
subsequences, and the part of these subsequences that is weakly on the left of 
n is in the same position in each permutation belonging to the class. 


DEFINITION 5.25 The blocks C and C" are called similar if their 
permutations have fundamental subsequences that are identical as permutation 
patterns. 


Thus in this case there is no requirement that the left part of the funda- 
mental subsequences must be in identical positions. 


Example 5.26 

The blocks containing the permutations 34152 and 42518 are similar. 
Indeed, permutations in the first block have fundamental subequence 152, 
and permutations in the second block have fundamental subsequence 253. 


5.1.5.1 Inductive Proof 


Our proof of Theorem 5.22 will be an induction on r. 


5.1.5.2 Initial Step 


To make our argument easier to follow, we introduce some new notions: 


DEFINITION 5.27 Let q be a pattern. Then to insert the entry y to 
the jth position of q is to put y between the 7 — 1st and the jth entry of q and 
to increase the value of any entry v for which originally v > y held by 1. 

The deletion of an entry is obtained similarly: erase the entry and decrease 

all entries larger than it by 1. 


LEMMA 5.28 
Let q be any pattern of length k. Then the number Cy(n) of 132-avoiding n- 
permutations that end with a subsequence of type q is a P-recursive function 


of n. 


PROOF _ The proof is by induction on k. If k = 0, then C,(n) = C, = 
7") /(n +1), the nth Catalan-number. We say in Example 5.2 that C;, is a 
P-recursive sequence, so the initial case is proved. 
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Let us now assume that we know that the statement is true for all subse- 
quences of length & — 1. We will prove it for the subsequence gq, which has 
length k. 

If g is not 132-avoiding, then clearly Cj(n) = 0. So we can suppose that 
is 132-avoiding. Now we consider two separate cases. 


(a) If the first element q, of q is not the smallest one, then the entry 1 of 
our n-permutation p cannot be on the left of q,, thus in particular, the 
entry 1 of p is one of the last & entries, which form a subsequence of type 
q. Then it must be the smallest of these last k entries, thus we know 
exactly where the entry 1 of p is located. Let us delete the smallest 
entry of q to get the subsequence q’. Apply the induction hypothesis 
to q’ to get that Cy (n — 1) is P-recursive. Then insert 1 to its original 
place to see that C,(n) is P-recursive. 


(b) If qi is the smallest element of g, then it is easy to apply what we have 
just shown in the previous case. Let q’’ be the subsequence obtained 


from q by deleting qi. Moreover, let r2,173,--+ ,r~ (resp.) be the subse- 
quences whose last k — 1 elements determine a subsequence of type q”’ 
and whose first elements (resp.) are 2,3,--- ,&. Then it is obvious that 


k 
Colt) = Cr (n) — Ca, (n) (5.11) 


The first term of the right-hand side is P-recursive by induction and 
the second one is P-recursive by the previous case, and the lemma is 
proved. 


This completes the proof. We will refer to the method applied in case (b) as 
the complementing method. | 


COROLLARY 5.29 

Let q be a pattern of length k. Then the number K,(n) of 132-avoiding n- 
permutations in which the k largest entries form a subsequence of type q is a 
P-recursive function of n. 


PROOF This is true as Kg(n) = Cy-1(n). Indeed, taking inverses 
turns the last k entries of a permutation p into the k largest entries of the 


permutation p7!. 


5.1.5.3 Induction Step 


Now would be the time to carry out the induction step of our inductive proof. 
It turns out that this is easier to do if we generalize our statement even further. 
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THEOREM 5.30 

Let q be any pattern of length k. Then the number Cy,-(n) of n-permutations 
that contain exactly r subsequences of type 132 and end with a subsequence of 
type q is a P-recursive function of n. 


Note that Theorem 5.22 is a very special case of Theorem 5.30. Also note 
that equivalently we can state the theorem for the numbers Ky,,(n) of n- 
permutations that contain exactly r subsequences of type 132 and in which 
the largest k entries form a subsequence of type q. 


PROOF Induction on r. If r = 0, then Theorem 5.30 reduces to Lemma 
5.28 and Corollary 5.29. 

The inductive step will be carried out in a divide and conquer fashion. For 
brevity, we call the statement of Theorem 5.22 the weak statement and we call 
the statement of Theorem 5.30 the strong statement. 

For our induction proof, we have to show that the strong statement for r—1 
implies the strong statement for r. We do this in two parts. 


(a) We show that if the strong statement holds for all non-negative integers 
at most as large as (r — 1), then the weak statement holds for r. 


(b) Then we prove that if the weak statement holds for r, then the strong 
statement holds for r. 


Let us follow up on these promises. 


(a) Let us assume now that the strong statement holds for all non-negative 
integers at most as large as r—1. Choose any block C of n-permutations. 
Suppose the fundamental subsequence type of C' contains exactly s sub- 
sequences of type 132, where s < r. 


e Let s > 1. How can a permutation in C' contain 132-patterns that 
are not contained in the fundamental subsequence? Clearly, they 
must be either entirely before the entry n or entirely after it. If 
there are 7 such subsequences before n and j such subsequences 
after n, then i+7+s =r must hold. Denote by q, the pattern 
of all front entries in the fundamental subsequence and by qo the 
pattern of all back entries there. Then with the previous notation 
we have 


f(n1,n2, 91,92; 4, 9, s) oa Cy i(m1) . Kq,j(n2) 


such permutations, where n (resp. m2) denotes the number of 
front (resp. back) entries which are not in the fundamental subse- 
quence. Indeed, entries of the fundamental subsequence are either 
the rightmost front entries, or the largest back entries. We know 
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by induction that Cy,,i:(n1) is P-recursive in n; and Kq,,;(n2) is 
P-recursive in ng. Therefore, their convolution 


f(n,q.94,5,8)= So f(mi,ne.q,92.4,9,8) (5.12) 


nirn2g=n 


= ys Cy i(m1) ‘ Kq,j(n2) (5.13) 


nirn2g=n 


is P-recursive in n. Clearly this convolution expresses the number 
of n-permutations with exactly r subsequences of type 132 in all 
blocks similar to C. 

It is clear now that we have only a bounded number of choices for 
i, j, and s so that i+ 7 +s = 7, thus we can sum (5.12) over all 
these choices and still get that 


f(n,m, 92) =S°f(n,0, 0,4, 5,8) (5.14) 


i,j, 


is P-recursive in n. (Recall that s > 0, thus we can always use the 
induction hypothesis.) Summing (5.14) over all q; and q2 we get 


that 
f(n) = S> f(n, a1, 92) (5.15) 


1,42 


As in this section, the only pattern we study is 132, we simplify 
our notations a little bit by writing S,(n) for S,,,(132). 


Now suppose s = 0. Then any 132-subsequence must be either 
entirely on the left of the entry n or entirely on the right of n. 
Moreover, the position of n completely determines the set of the 
front and back entries. If n is in the 7-th position, and we have j 
132-subsequences in the front and r— in the back, then this gives 
us 

g(t, J) = S(t — 1) Sp_j(n — 7) (5.16) 


permutations of the desired kind. If 1 < 7 < r—1, then the induc- 
tion hypothesis applies for S; and S,_,;, therefore, after summing 
(5.16) over all 2, 


a(n) = > ji -1)$,-s(n- 7) (5.17) 


is P-recursive in n. If 7 = 0 or 7 = r, then we cannot apply 
the induction hypothesis. By a similar argument as above we get 
nevertheless that in this case we have 


2-S°S,(i- Cri (5.18) 
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ee 


n-permutations with exactly r 132-subsequences. (We remind the 
reader that So(n—7) = Avn_;(132) = Cy_i, the (n —7)th Catalan- 
number.) 


Summing (5.15), (5.17), and (5.18), we get 


Let F, G, C, and S;. denote the ordinary generating functions of f(n), 
g(n), Cn, and S,(n). Then the previous equation yields 


S,(z) = F(z) + G(z) + 2z- C(z)S(z), 


that is, 
F(z) + G(z) 


A ae aD PRTOIE 


(5.20) 
Therefore, S,(z) is d-finite as it is the product of two d-finite power 
series. Indeed, the numerator is d-finite and 1/(1—2zC(z)) = 1/V1— 4z 
is d-finite as it is algebraic. Thus, S,(n) is P-recursive and we are done 
with the first part of the proof. 


Now we prove that the weak statement for r implies the strong statement 
for r. Let q be any subsequence of length k. We must prove that 
the number C,,-(n) of n-permutations that end with a subsequence of 
type q and contain exactly r subsequences of type 132 is a P-recursive 
function of n. Clearly, if g contains more than r 132-subsequences, then 
Cy,r(n) = 0 and we are done. Otherwise, we will do induction on k, the 
case of k = 1 being obvious. There are three different cases to consider. 


e If g has more than r inversions, then it is obvious that no such 
permutation can have its entry 1 on the left of the last k elements. 
Therefore, this entry 1 must be a part of the g-subsequence formed 
by the last & elements. Now, deleting this entry 1 we may or may 
not lose some 132-patterns, as there may or may not be inversions 
on its right, but we can read off this information from q. (See the 
next example.) Again, let qg’ be the pattern obtained from q by 
deleting its entry 1. If we do not lose any 132-patterns by this 
deletion, then we are left with an (n — 1)-permutation ending with 
the pattern gq’ and having r subsequences of type 132. If we lose t 
such patterns, then we are left with an (n — 1)-permutation ending 
with q’ and having r — t such subsequences. Both cases give rise 
to a P-recursive function of n by our induction hypothesis as q’ is 
shorter than q. 
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Example 5.31 


If ¢g= 341652, then q’ = 23541; thus we lose three subsequences 
of type 132 when deleting 1. Therefore, we can apply our inductive 
hypothesis for r — 3, then reinsert the entry 1 to its place. If q = 
3124, then we do not lose any 132-patterns when deleting the entry 
1 and getting q’ = 213. Thus we still need to count permutations 
with r 132-patterns, but they must end with q’, not with q. The 
pattern q’ is shorter than q, thus the induction hypothesis on k can 
be applied. 


e If q has at most r inversions, but qg is not the monotonic pattern 
12.---k, then it can also happen that the entry 1 is not among the 
last k entries of our permutation. However, we claim that it cannot 
be too far away from them. Indeed, let y be an element from the 
last k elements of the permutation (thus one of the elements of the 
set L of the last k entries that form the ending q) that is smaller 
than some other element x € L on its left. Then clearly, ifn is large 
enough, then y must be smaller than r++ 1; otherwise, we would 
have too many 132-patterns of the form way. So y is bounded. 
If the entry 1 of the permutation were more than 2r+k+1 to 
the left of y, then there would necessarily be more than r elements 
between 1 and y that are larger than y, a contradiction. Thus the 
distance between 1 and y is bounded. Therefore, we can consider 
all possibilities for the position of the entry 1 of the permutation 
and for the subsequence on its right. In each case we can delete the 
entry 1 and reduce the enumeration to one with a smaller value of r 
(as q has at least one inversion), then use the inductive hypothesis 
on r. Thus this case contributes a bounded number of P-recursive 
functions, too. 


e Finally, if g is the monotonic pattern 12 --- k, then use the com- 
plementing method of Lemma 5.28. 


Now note that speaking in terms of ordinary generating functions, all opera- 
tions we made throughout the induction step were either adding or multiplying 
a finite number of power series together. In particular, the ordinary generat- 
ing function C(z) of our initial C,-sequence (that is, when r = 0 and k = 0) 
is C(z) = tt. thus an algebraic power series. Therefore, the ordinary 
generating function of S,(n), the power series S;.(z), is algebraic, too. 

Now note a bit more precisely that throughout our proof we have either 
added formal power series together, or, as in (5.16), multiplied two functions of 
type S;(i—1) together, or, as in (5.20), multiplied a power series by the power 
series 1/(1 — 2z- C(z)) = 1/V1-—4z. Therefore, the following proposition is 


immediate: 
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PROPOSITION 5.32 

Let r > 1, and recall that S,.(z) denotes the ordinary generating function of 
the numbers S,.(n). Then S,(z) € Cl[z, V1 — 4z]]. Moreover, when written in 
smallest terms, the denominator of S(z) is a power of (/1— 4z). 


It is convenient to work in this setting as the square of \/1 — 4z is an element 
of C[z], which makes computations much easier. 

We are going to determine the exponent f(r) that /1 — 4z has in the de- 
nominator of G,(z). Equations (5.16) and (5.20) show that 


f(r) = manisicr(f(i) + f(r —7t)) +1. (5.21) 


We now claim that f(r) = 2r—1. It is easy to compute (see [47]) that 
f(1) = 1. Now suppose by induction that we know our claim for all positive 
integers smaller than r. Then (5.21) and the induction hypothesis yield that 
for some i, we have 


fir) =(f@+ f(r -—2) +1 = Qi-1)4 Qr- 2-1) +1= 2r-1, 


which was to be proved. 

Recall now that 1/VT—4z = D5, (7")2” and that the sequence (*") 
satisfies a linear recursion. Differentiate both sides of this equation several 
times. On the left-hand side, each differentiation will add two to the exponent 
of V1 — 4z in the denominator. On the right-handvside, it will add one to the 
degree of the highest-degree polynomials appearing in the recursive formula for 
the coefficients. Thus, differentiating r—1 times we get that the denominator 
of S,(z) gives rise to a polynomial recursion of degree r. The numerator of 
S,-(z) cannot increase this degree. 

We collect our observations in our last lemma: 


LEMMA _ 5.33 
Letr > 1 and let S;.(x) be the generating function for the sequence { Sy,,(132)}n.- 
Write S,.(z) in lowest terms. Then the denominator of S;.(z) is equal to 


(6/1 —4z)?"-* = (1— z)?-+ 4/1 — 4e. 


Therefore, the sequence {Sp,-(132)}, satisfies a polynomial recurrence relation 
with degree at most r. 
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5.2 Containing a Pattern Many Times 
5.2.1 Packing Densities 


In this section we will turn around and instead of trying to avoid a pattern, or 
containing a pattern just a few times, we will try to contain a given pattern q 
as many times as possible. More precisely, for fixed g and n, we want to find 
the largest integer M,,,q so that there exists an n-permutation that contains 
exactly Mn,q copies of g. However, for divisibility or other reasons, it can 
happen that not all integers n behave in the same way as far as containment of 
many copies of g is concerned. Therefore, we would like to have a more global 
way of measuring how many copies of g can be packed into an n-permutation. 
The following definition establishes this measurement. 


DEFINITION 5.34 Let q be a pattern of k elements. Then the packing 
density of g, denoted g(q), is given by 


g(q@) = lim —* 


In other words, the packing density of q tells us how large a portion of 
all k-element subwords of an n-permutation can be copies of q as n goes to 
infinity. 

The alert reader probably caught us red-handed in committing the sin of 
Mn4 to define packing density, without proving first that that 


using limy +o = 
k 

limit exists. The limit does exist, however, as can be seen from the following 

result of Galvin, which appeared in [268]. In what follows, if an n-permutation 

p contains M(n,q) copies of g, then we will say that p is a q-optimal permu- 

tation. 


LEMMA 5.35 
Let q be a pattern of length k. Then forn > k, the sequence 


. . . : M . 
is nonincreasing, therefore limn—oo () exists. 
k 


PROOF Let g(n,q) = Mn,q/ (it). We show that 


g(n,q) S g(n— 1,4), 


which will clearly imply our claim. 
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Let p be an n-permutation that is g-optimal, that is, contains a maximal 
number of copies of g. Let pi), p(2),--* , Pin) denote the n permutations of 
length n — 1 that we obtain from p by omitting the first, second, and so 
on, entry of p. It is clear that the proportion g(n,q) of q-copies among all 
k-element subwords of p is the average of the proportions prop(p,q,i) of q- 
copies among all k-element subwords of the p,;). (The average is taken over 
all i € [n].) There has to be at least one i for which prop(p, q,7) is not below 
that average, that is, for which 


1 Prop(P, 4,5) 
nr 


g(n,q) = < prop(p, q,%) <9(n— 1,9). 
This shows that our sequence is nonincreasing, and therefore, consisting of 
nonnegative elements, convergent. 


If g is a monotone pattern of length k, then the answer is trivial. Indeed, we 
have Mn .q = a and this maximum is attained on the monotone permutation 
of length n. Therefore, the packing density of any monotone pattern is 1. 
Hence, for the remainder of this section, we will assume that q is not monotone 
(unless otherwise stated). 


5.2.2 Layered Patterns 


For a general pattern, however, the question of determining M,,,q is still un- 
solved. However, for layered patterns, which we introduced in Definition 4.94, 
we can say significantly more. This is because of the following unpublished 
theorem of Walter Stromquist. While Stromquist has never published his 
proof, it can be found in [268]. 


THEOREM 5.36 
Let q be a layered pattern. Then there exists a layered permutation p so that 
p has exactly My,q copies of q. 


That is, among the n-permutations that are q-optimal, there is at least 
one layered permutation. The number of layered permutations of length n is 
obviously 2”~!, which is much less than n!, so the above theorem is extremely 
useful if we want to find a g-optimal permutation with the help of a computer 
program. 

We point out that while it is not true in general that all g-optimal permu- 
tations are layered, there are special cases when they are. See Exercise 27 for 
a large special case. 

What can we say about the structure of a layered g-optimal permutation 
when q is layered? Even this question is unsolved in this generality. However, 
there are certain precise answers in the case when q has only two layers. We 
will discuss the shortest of these patterns, the pattern gq = 132. For the more 
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general case of g = 1k(k — 1)---32, see [55]. For a different, and also more 
general treatment, see [268]. 


5.2.2.1 Pattern 132 


We want to construct an n-permutation that contains 132 as many times as 
possible. Our intuition might suggest that (disregarding questions of divisi- 
bility) we split the set [n] into three parts, then place entries from 1 to n/3 
into the first n/3 positions, place entries from } + 1 to on into the last n/3 
positions, finally place entries from a +1 to n into the middle n/3 positions. 
Then we apply the same strategy recursively in each of the three tiers we 
have just created. This construction imitates the pattern 132 quite closely, 
so it seems reasonable to conjecture that it is optimal. Nevertheless, this 
conjecture is false. For instance, if n = 9, then this method constructs the 
permutation 132 798 465 that has 36 copies of 132 (27 copies containing one 
entry from each tier, three copies within one tier, and six copies containing 
one entry from the first tier and two entries in decreasing order from one of 
the other tiers). On the other hand, the permutation 1 32 987654 contains 46 
copies of 132. Indeed, choosing one of the three leftmost entries and two of 
the six rightmost entries in 3- () = 45 ways we get a copy of 132, and we also 
get a copy of 132 if we select the three leftmost entries. Therefore, we have 
to look for 132-optimal permutations with other methods. 

Let p be a 132-optimal n-permutation that is layered. We know from The- 
orem 5.36 that such a permutation exists. We will show that layered 132- 
optimal permutations have a simple recursive structure. This is because of 
the fact, which we will use many times, that to form a 132 pattern in a lay- 
ered permutation one must take a single element from some layer and a pair 
of elements from a subsequent layer. 


PROPOSITION 5.37 
Let p be a layered 132-optimal n-permutation whose last layer is of length m. 
Then the leftmost k =n—m elements of p form a 132-optimal k-permutation. 


PROOF _ Let D, be the number of 132-copies of p that are disjoint from 
the last layer. The number of 132-copies of p is clearly k('’) + Dx. So once k 
is chosen, p will have the maximum number of copies if D, is maximal. (Note 
that this argument works even if m = 1 as () = 0.) | 


We point out that the proof of this proposition uses the fact that 132 has 
only two layers, the first of which is a singleton. 

As we are now concentrating on copies of 132, we simplify our notation by 
setting M, = Mn132. That is, 4, denotes the maximum number of 132- 
copies an n-permutation can have. Then the previous proposition implies 
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Mp = max (on ()) (5.22) 


The integer k for which the right-hand side attains its maximum will play a 
crucial role throughout this section and the next one. Therefore, we introduce 
specific notation for it. 


that 


DEFINITION 5.38 For any positive integer n, let ky, be the positive 


integer for which 
My, = max (1 4 (3) 
k<n 2 


is maximal. If there are several integers with this property, then let ky, be the 
largest among them. 


In other words, k,, is the largest possible length of the remaining permu- 
tation after removing the last layer of an optimal n-permutation p. When 
there is no danger of confusion, we will only write & instead of k,,, to simplify 
notation. We will also always use m = n — k to denote the length of the last 
layer of p. 

We continue our search for 132-optimal n-permutations. The construction 
at the beginning of this section, which was shown not to be optimal, leads to 
n-permutations with 


n\3 ny\3 Fi n\3 nd iy ae 
eg re cea) ~ 29 OF 


copies of 132. Our goal is to find a construction, for general n, that provides 
more copies. The proof of the following lemma provides such a construction. 


LEMMA 5.39 

The inequality 

Mn 
—~ > 2V3—3 = 0.464 


holds. 


PROOF Let us look for 132-optimal n-permutations in a specific form, 
which we will call geometric form. In these permutations, disregarding ques- 
tions of divisibility, the last layer is of length m = t-n (for some t < 1), 
the penultimate layer is of length (1 — t)tn, the one before that is of length 
(1 — t)?tn, and so on. In other words, the layer lengths form a geometric 
progression with quotient 1 — t. 

How many 132-patterns will such a permutation p(t) contain? It follows 
from the layered structure of p(t) that in a copy of 132 in p(t), the last two 
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entries will come from the same layer. We will count the 132-copies according 
to the layers that contain their last two entries. 

There are ("})k = 5 -¢?(1 —t)n? — O(n?) copies of 132 that have their last 
two entries on the last layer of p(t). Similarly, there are roughly $-¢?(1—t)*n3 
copies of 132 that have their last two entries on the penultimate layer. There 
are roughly $-¢?(1—t)'n® copies that have their last two entries on the layer 
before that, and so on. Therefore, if n is large enough, the n-permutation p(t) 
created in this way will have 


n3t?(1—t) 


HAH On oer) (6.23) 


© 2) 21-9 — O(n? logn) = 


j21 


copies of 132. Indeed, there are at most O(log n) layers, and on each layer, 
the error term is at most O(n”). Note that, for instance, for t = 2/3 (this is 
what we used in our counterexample above), the above formula shows that 
we have n?/13 — O(n? logn) copies of 132, which is significantly more than 
the n3/24 achieved by our first construction. However, t = 2/3 is still not the 
best choice. To find the best choice for t, we simply use elementary calculus 
to find the maximum of the function 


7 (1 _ t) _ t(1 - t) 
MO = sq Gps * 343): 


We find that the maximum of f on the interval [0,1] is taken at t = a-V3 ~) 
0.634, so this is the best choice for t. 
Our construction works for all n, proving that 


t(1-t) 2 
M, >So" - : 
> We B43)” O(n“ log n) 


It is then clear from the definition of g(132) that 


M, : t 
g(132) = lim ——> lim max ft) = 2/3 — 3 = 0.464. (5.24) 
n—->0o Gi n—-oo tE[0,1] 1/6 


We claim that in the long run, nothing beats patterns in geometric form. 
In other words, the above construction is asymptotically optimal among all n 
permutations. By this we mean that the weak inequality in (5.24) is actually 
an equality. This is the content of the following theorem. 


THEOREM 5.40 
[268] The above construction is asymptotically optimal, that is, 


g(132) = 2/3 — 3 = 0.464. 
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PROOF _ Let 


32z(1 — z) 
B= ———“~-“= h(z). 5.25 
Po ieepee eon 2) 


We will first prove that B = g(132), then we will prove that 


=6 
eT) 
which will complete the proof of our theorem by (5.24). To prove the first 
claim, it suffices to prove that 


M 1 
—_~<B+O (=) : (5.26) 
(5) n 
or, in other words, that for all n, we have 
3 
n 
M,<B re 
We are going to prove this claim by induction on n. For n = 1, the statement 
is true as the left-hand side is equal to 0. Now assume that we know the 
statement for all positive integers less than n. 
Recall that we have described the structure of 132-optimal permutations in 
Proposition 5.37, and in the subsequent formula (5.22). Therefore, we have 


Mg =mpe (3, +2()) (21 


= pc (ane+e()) <anc(Mer hn) (6.28) 
= max (5 + a) (5.29) 
= max (= (Bii-2)? +30 -— ne). (5.30) 


In the last step, we dropped the requirement that tn = m and (1—t)n =k 
be integers. 

Remember that B was defined as a maximum in (5.25). From that definition 
it follows that 


32z(1 — z) 

I oes 

3z(1— z)? < B(1—z?) 
Bz? + 3z(1— z)* < B, 


or, setting f= 1-2, 


Bil —t)? +3(1 — tt? < B. 
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Comparing this last equation with the last term of (5.30) proves (5.26). So, 
we have shown that g(132) = B. 
Therefore, our theorem will be proved if we show that B = 6 max;¢j9,1) f(t). 


This is immediate if we observe that h(z) = 6f(1— z). | 


As this is our first result in the theory of packing densities, it is worth 
pointing out the interesting fact that while our problem is about integers, the 
best ratio of consecutive layer lengths in order to contain many copies of 132 
was provided by an irrational number, the number Vaa1 

Most results on packing densities are about layered patterns, which is not 
surprising since Theorem 5.36 is a very powerful tool in studying permutations 
containing a maximum number of copies of such patterns. A sampling of 
these results is contained in the Exercises. We point out that Problem Plus 
21 contains an interesting result about a non-layered pattern. 


Da 


5.3 Containing a Pattern a Given Number of Times 


In the last section, we looked for permutations that contained a given pattern 
as many times as possible. In this section, we will take a different approach 
and we will be looking for permutations that contain a given pattern a given 
number of times. 


DEFINITION 5.41 Letn be a fixed positive integer, and let q be a fixed 

pattern. Let Sp-(q) be the number of n-permutations with exactly c patterns 
of type q. The sequence {S,,-(q)}c>o0 is called the frequency sequence of the 
pattern q for n. 


The alert and meticulous reader will point out that we have already seen 
an example for a frequency sequence, and we have proved several interest- 
ing statements about it. Indeed, inversions, our main topic in Section 2.1, 
are nothing but 21-patterns. We have seen that the frequency sequence of 
inversions is log-concave, and is therefore unimodal. 

When q is longer than 2, numerical evidence suggests that the frequency 
sequence of q will no longer be unimodal, let alone log-concave. In fact, 
disturbing objects called internal zeros seem to be present in most frequency 
sequences. 


DEFINITION 5.42 An integer c is called an internal zero of the sequence 
{Sn,c(Q}e>o if we have S,,,-(q) =0, but there exist cy and cy so that 


(a) cr <C< 2, and 
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(b) Onset (q) > 0, and ries (q) > 0. 


Example 5.43 
Let gq = 123, and let n > 3 be any positive integer. Then c = (3) —lis 


an internal zero of the sequence (Sp,c(q))e>0 as we can have c; = (57) and 


Q= (3) for the permutations n12---n —1 and 123---n, respectively. 0 


5.3.1 Construction with a Given Number of Copies 


In this subsection we show that there are infinitely many integers n so that 
the sequence F}, = Sy,-(132) does not have internal zeros. We will call such an 
integer, or its corresponding sequence, NIZ (no internal zero), and otherwise 
IZ. Our strategy is recursive; we will show that if k, is NIZ, so is n, where 
ky, was described in Definition 5.38. As ky, <n, this will lead to an infinite 
sequence of NIZ integers. There is a problem, however. In order for this 
strategy to work, we must ensure that given k, there is an n such that k = ky. 
This is the purpose of the following theorem. 


THEOREM 5.44 
The sequence {kn}n>1 diverges to infinity and satisfies 


kn <kn41 < kn +1 


for alln > 1. So, in particular, for all positive integers k there is a positive 
integer n so that kn =k. 


The next subsection is devoted to a proof of this theorem. We suggest that 
the reader assume the result now and continue with this section to preserve 
continuity. Keep the notation M; from the previous section. Before starting 
the proof of our main theorem, we need only note the useful fact that 


My, > (“S ') (5.31) 


which follows by considering the permutation 1k(k — 1)(k — 2)---32. 


THEOREM 5.45 
There are infinitely many NIZ integers. 


PROOF By Theorem 5.44, it suffices to show (for sufficiently large k,,) 
that if k, is NIZ, then so isn. By the same theorem, we can choose n so that 
ky is NIZ, and k, > 4. To simplify notation in what follows, we will write k 
for ky. 
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Now givenc withO <c< M, = Mr+k(‘3), we will construct a permutation 
p € S, having c = c(p) copies of 132. Because of (5.31) and k > 4, we have 
M;, > k—1. So it is possible to write c (not necessarily uniquely) as c = ks +t 
withO<s< (3) and 0 <t< Mx. Since k is NIZ, there is a permutation 
p’ € S, with c(p') =t. Also, it is not difficult to prove (see Exercise 15) that 
there is a permutation in S,, with no copies of 132 and s copies of 21. Let 
p” be the result of adding k to every element of that permutation. Then, by 
construction, p = p’p” € S, and c(p) = ks + t = c as desired. 


One can modify the proof of the previous theorem to locate precisely where 
the internal zeros could be for an IZ sequence. We will need the fact (estab- 
lished by computer) that for n < 12 the only IZ integers are 6, 8, and 9, and 
that they all satisfied the following result. 


THEOREM 5.46 
For any positive integer n, the sequence F,, does not have internal zeros, except 
possibly forc = M, —1 orc = M, — 2, but not both. 


PROOF ~~ We prove this theorem by induction on n. Numerical evidence 
shows that the statement is true for n < 12. Now suppose we know the 
statement for all integers smaller than n, and prove it for n. If n is NIZ, then 
we are done. 

If n is IZ then, by the proof of Theorem 5.45, k = k, is IZ. So k > 6 and 
we have M, > k +2 by (5.31). Now take c with 0 < c < M, — 3 so that we 
can write c= ks+twith0O<s< ) and 0 < t < M;, —3. Since the portion 
of Fy up to Sx,132(Mz — 3) has no internal zeros by induction, we can use the 
same technique as in the previous theorem to construct a permutation p with 
c(p) = c for c in the given range. Furthermore, this construction shows that 
if Spi32(M;, — i) 4 0 for i = 1 or 2 then S;,132(Mn — 7%) 4 0. This completes 


the proof. 


5.3.2 Sequence {kn}n>o 


In order to prove Theorem 5.44, we first need a lemma about the lengths of 
various parts of a 132-optimal permutation p. In all that follows, we use the 
notation 
b = length of the next-to-last layer of p 
a = length of the string created by removing the last two layers of p 
=n—-—m—b 
=k-—b. 
The following lemma summarizes some innocent-looking properties of the 
above layer lengths that we will need in our proof of Theorem 5.44. As the 
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reader will see, these properties are actually not all as immediate as they look. 


LEMMA 5.47 
The following inequalities hold: 


(i) b<m, 
(ii) a< (m—1)/2, 
(iti) m>k, which implies m > n/2 and k < n/2, 
(iv) m < 2(n + 1)/3. 


PROOF The basic idea behind all four of the inequalities is as follows. 
Let p’ be the permutation obtained from our 132-optimal permutation p by 
replacing its last two layers with a last layer of length m’ and a next-to-last 
layer of length b’. Then in passing from p to p’ we lose some 132-patterns and 
gain some. Since p was optimal, the number lost must be at least as large as 
the number gained. And this inequality can be manipulated to give the one 
desired. 

For the details, the following chart gives the relevant information to describe 
p’ for each of the four inequalities. In the second case, the last two layers of 
p are combined into one, so the value of b’ is irrelevant. 


m’ | b’ |number of gained 132-patterns < number of lost 132-patterns 


(51) tab < (a+b)(m—1) 


Now (i) and (ii) follow immediately by canceling bm from the inequalities in 
the first two rows of the table. From these two, it follows that a(b—1) < ae 
So using the third line of the chart 


(k—1)m=(atb-1)m< & +a(b—1) <2(7) =m(m—1) 
and cancelling m gives (iii). For (iv) we have 


("5 ) (Mp) Fabs +0) = 1) = (n= myn = 2), 
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Canceling m — 1 and solving for m completes the proof. | 


We now turn to the proof of Theorem 5.44. First note that, by Lemma 5.47 
(iv), we have 
n—2 

7 
So {kn}n>1 clearly diverges to infinity. For our next step, we prove that 
{kn }n>1 is monotonically weakly increasing. Let p,,; denote an n-permutation 
whose last layer is of length n—7, and whose leftmost 7 entries form an optimal 
i-permutation, and let Cn; = C(pn,i). Clearly 


n—-t 
ni = Mi +1 ‘ 
Cn, +if 9 ) 
PROPOSITION 5.48 
For alln > 1, the inequality ky < ky41 holds. 


k=n-m> 


PROOF As usual, let k = k,. Then it suffices to show that cni14 > 
Cn+1,i for alla < k. This is equivalent to showing that 


l—k iH 4 
meen ("tS Jemri("* ) (5.32) 


However, by the definition of k, we know that for all i < k, 
n—-k [n-4 
Mh +h 9 Jem si( 9 i: (5.33) 


Comparing (5.33) and (5.32), we see that all we need to do is to prove the 
inequality k(n — k) > i(n—7). Rearranging terms in order to cancel k — i 
gives the equivalent inequality n > k +7. However, k < n/2 by Lemma 5.47 
(iii), and so the bound on i gives k +1 < 2k <n. 


The proof of the upper bound on k,,+1 is a bit more involved but follows the 
same general lines as the previous demonstration. Note that this will finish 
the proof of Theorem 5.44. 


LEMMA 5.49 
For all positive integers n, the inequality ky < kn41 < kn +1 holds. 


PROOF We are going to use induction on n. The statement is easy 
to check for n < 2. Suppose we know that the lemma is true for integers 
smaller than or equal to n, and prove it for n+ 1. For simplicity, set k = kp, 
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m=n—k, and ¢ = ¢n414. Since we have already proved the lower bound, 
it suffices to show that 


1 
Ci => Ci41 fork+1<i< | ‘ (5.34) 


Note that we do not have to consider 4 > |(n + 1)/2] because of Lemma 5.47 
(iii). 

We prove (5.34) by induction on 7. For the base case, i = k +1, we wish to 
show 


Meat (k+1) (") > Mnzo + (k +2) i ‘). (5.35) 


However, since py, is optimal by assumption, we have 
-—1 
M+ () > Metit @+0(") i: (5.36) 


Comparing (5.36) and (5.35) and rearranging terms, it suffices to prove 


m—1 > (Mp2 — Mpgii) — (Mr+i — Mg). (5.37) 


vt 


Let p’ € Sz, p” © Seyi, and p'” € Sz42 be layered 132-optimal permutations 
having last layer lengths m’, m”, and m’”, respectively, as short as possible. 
Since n > 2 and k < n/2, we have k+2 <n and so, by induction, these three 
permutations satisfy the lemma. If m’” = m’ +1 then let x be the largest 
element in the last layer of p” (namely « = k+ 1). Otherwise, m” = m’ and 
removing the last layer of both p’ and p” leaves permutations in S$; and 
Sk—m’+1, respectively. We can iterate this process until we find the single 
layer where p’ and p” have different lengths (those lengths must differ by 1) 
and let x be the largest element in that layer of p”. Similarly we can find the 
element y, which is largest in the unique layer where p” and p””’ have different 
lengths. 

Now let 


r = number of 132-patterns in p”’ containing neither x nor y, 
s = number of 132-patterns in p’” containing x but not y, 

t = number of 132-patterns in p’”’ containing y but not x, and 
u = number of 132-patterns in p’”” containing both x and y. 

wy 


Note that there is a bijection between the 132-patterns of p’” not containing 
y and the 132-patterns of p’. A similar statement holds for p” and p’. So 


My=7, Mrsi=rt+s, Mizg=r+sttt+u. 


Note also that s >t because increasing the length of the layer of x results in 
the greatest number of 132-patterns being added to p’. It follows that 


(Mr+2 _ Mp+1) _ (Mr+i _ Mx) =t+u-—s<u. 
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However, there are only k elements of p’” other than x and y, sou <k<m-1 
by Lemma 5.47 (iii). This completes the proof of (5.37) and of the base case 
for the induction on i. 

The proof of the induction step is similar. Assume that (5.34) is true for 
i—1 so that 


I+1 l 
Mi hG= ( : ) > M, +i(). (5.38) 
where 1 =n-+1-—i. We wish to prove 
fl . I-11 
M;+% 9 > Mi4i + (i + 1) 9 3 (5.39) 


Comparing the last two ineqaualities as usual and simplifying, we need to 
show 
21—4-1> (Mi4i — M;) — (Mi; — Mi-1). 


Proceeding exactly as in the base case, we will be done if we can show that 
2l1—i—1 > i—1 or equivalently 1 > 7. However, this is straightforward because 
l=n+1-iandi < |[(n+1)/2]. 


We have seen that there are infinitely many NIZ integers. It is natural to 
ask whether there are infinitely many IZ integers as well. The answer is in 
the affirmative, and can be found in [55]. 


a 


Exercises 
1. Find an explicit formula for 5)32,1(7). 


2. Prove that the number of ways to dissect a convex (n+ 1)-gon into n—2 
parts with noncrossing diagonals is S132,1(n). 


3. (+) Generalizing the previous problem, find a formula for the number 
f(n,d) of ways to dissect a convex n + 2-gon by d non-intersecting di- 
agonals. 


4. Let f, and gy be two sequences that differ only in a finite number of 
terms. Prove that f,, is P-recursive if and only if g, is P-recursive. 


5. Prove that for any fixed k, the sequence {$(n,k)}, is P-recursive. 


6. Prove that for any fixed k, the sequence {P(n,k)}, is P-recursive. See 
Exercise 33 of Chapter 1 for the definition of P(n, k). 


7. Let A(z) and B(z) be two algebraic power series. Prove that A(z)B(z) 
and A(z) + B(z) are also algebraic power series. 
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8. 


16. 


17. 


18. 


19. 


20. 
21. 


22. 


23. 


. Prove that for any nonnegative integer s with s < ( 


Let pz(n) be the number of partitions of the integer n into at most k 
parts. Is py(n) a P-recursive sequence? 


. Prove that f(z) = sin z is d-finite, but not algebraic. 

. Prove that Av,,(3142, 4231) is a P-recursive sequence. 

. Prove that Av,,(1342, 2431) is a P-recursive sequence. 

. Prove that the sequence f(n) = (n!)” is not P-recursive. 

. (-) Characterize layered permutations by pattern avoidance. 


. Find the packing density of the pattern gq, = 1 (k+1)k---2. 


A there is a per- 


mutation p € S, having s copies of the pattern 21 and no copies of 
132. 


Let N be a positive integer. Show that there exists a pattern q and a 
positive integer n so that the frequency sequence (Sy,q(c))e>0 contains 
N consecutive internal zeros. 


(—) Let p be a 321-avoiding n-permutation. At most how many inver- 
sions can p contain? 


Let p be a k---321-avoiding n-permutation. At most how many copies 
of gq = (k —1)---321 can p contain? 


A k-superpattern is a permutation that contains all k! patterns of length 
k. Let sp(k) be the length of the shortest k-superpattern. For instance, 
sp(2) = 3, as 132 is a 2-superpattern of length three, and obviously, 
there is no shorter 2-superpattern. 
(a) Determine sp(3). 
(b) Prove that sp(4) 
(c) Prove that sp(k) 


Prove that sp(k) < (k—1)? +1. 


(a) Prove that sp(4) > 6. 
(b) Prove that sp(4) > 7. 


(a) Prove that if & is large enough, then sp(k) > a 
(b) Prove that for all integers k > 4, we have sp(k) > 2k. 
Let p € Sg. Then p has 28 = 256 subwords. Let S' be the set of these 


subwords. Prove that the number of distinct permutations patterns that 
occur in S' is at most 127. 
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24. 


25 


26. 
27. 


28. 


29. 


30. 


3l. 


32. 


33. 
34. 


35. 
36. 
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Find Mn,2143- 


Let q be a layered pattern consisting of k layers of length two each. Is 
it true that the g-optimal layered n-permutation consists of k layers of 
length n/k each? (We can assume that n is divisible by k.) 


Find all 1243-optimal n-permutations. 


(+) Let g be a layered pattern in which all layer lengths are at least 
two. Prove that all g-optimal permutations are layered. 


Prove that for any fixed k, the sequence G(n,k) of the numbers of n- 
permutations with k alternating runs is P-recursive. 


Let M,, be the number of lattice paths from (0,0) to (n,0) using steps 
(1,0), (1,1), and (1,—1) that never go below the line y = 0. (These 
lattice paths are called Motzkin paths. Prove that M,, is a P-recursive 
sequence. 


The numbers M,, enumerate many kinds of pattern-avoiding involutions, 
a fact to which we will return several times. 


Let r, be the number of lattice paths from (0,0) to (n,n) using steps 
(1,1), (1,0), and (0,1) that never go above the diagonal x = y. Prove 
that r, is a P-recursive sequence. What permutations are enumerated 
by these numbers? 


(—) Let allperm(n) be the smallest positive integer for which there exists 
a sequence S' of length allperm(n) whose elements are elements of the 
set [n] such that S' contains each of the n! permutations of length n as 
a subsequence (not necessarily in consecutive positions). 


For example, allperm(2) = 3, since S = 121 contains both 12 and 21 as 
a subsequence, and no sequence of length two has this property. 
Find allperm(3). 


Keeping the notation of the previous exercise, prove that allperm(n) < 
n? —n+A4. 


Prove that allperm(n) > (n+ 1)n/2. 


Prove that if p contains exactly one copy of 321, then p is not a derange- 
ment. 


Prove that for any k, the sequence Av,,(132,12---k) is P-recursive. 


Let us call a sequence s of elements of [n] an n-superpermutation if s 
contains all n! permutations of length n as a factor, that is, s contains 
each permutation p € S;, in consecutive positions. For instance, 121 is 
a 2-superpermutation, while 123121321 is a 3-superpermutation. Let 
spp(n) be the length of the shortest n-superpermutation. 
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(a) Prove that spp(n) > n!+n-—1. 
(b) Prove that the upper bound of part (a) cannot be achieved if n > 3. 


37. Prove that spp(n) < )77_, kl. 


38. Prove that spp(n) > n!+ (n—1)!4+n—-2. 


Problems Plus 


1. Find a formula for Sj23,1(), and prove from that formula that S123,1(n) 
is P-recursive. 


2. (a) Prove that the ordinary generating function of the sequence Av, (1234) 
is not algebraic. 
(b) Let k > 4 be an even positive integer. Prove that the ordinary 


generating function of the sequence Av,,(12---k) is not algebraic. 


3. Let S,i(n) denote the set of all permutations of length n that contain 
exactly 7 copies of the pattern g. Construct an injection from S321,1(n) 
to Avn+2(231). 


4. Construct an injection from S3216r,1(n) to Avozier(n), where r is any 
pattern. 


5. Let r be any pattern. Prove that the ordinary generating function of 
the sequence S32167,1(7) is not rational. 


6. Let k > 2 be an even integer. Prove that the ordinary generating 
function of the sequence $(%—1)...21,1(”) is not algebraic. 


7. Prove that 


2n — 6\ n® + 17n? — 80n + 80 
S132,2(n) = ( aaa 


n—A 2n(n — 1) 


8. Prove that 
2n ) 59n? + 117n + 100 


S123,2(n) = (, —4] In(2n—1)(n +5)’ 


9. Let 7 be a partition of the set [n]. We say that the 4-tuple of elements 
(a,b,c, d) is a crossing of a if a <b<c<d, and a and © are in a block 
m, of a, and b and d are in a different block 72 of a. Let r be a fixed 
natural number, and let H,-(n) be the number of partitions of [n] with 
exactly r crossings. Prove that H;,(n) is a P-recursive function of n. If 
possible, strengthen this claim. 
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11. 


12. 
13. 


14. 
15. 
16. 


17. 


18. 


19. 


20. 


21. 
22. 


23. 


24. 


Combinatorics of Permutations, Third Edition 


Let p be an n-permutation, and let f(p) be the number of all distinct 
patterns contained in p. For instance, if p = 1324, then f(p) = 7 for 
the patterns 1, 12, 21, 213, 123, 132, 1324. Let pat(n) = maxyes,, f(p). 
Prove that 

pat(n) 


i > 1. 
n>co 1.61” 


Let us keep the notation of the previous Problem Plus, and let us assume 
for simplicity that n = k?. Prove that pat(n) > 24@-”. 


Improve the lower bound for pat(n) given in the previous Problem Plus. 
Prove that 


2” —O (ater) < pat(n) < 2"-0 Ca) ‘ 


Find a 4-superpattern of length nine. 
Prove that there exists a k-superpattern of length She 


We call the permutation p a weak k-superpattern if for all patterns q of 
length k, at least one of q and q~! is contained in p. Prove that there 
exists a weak k-superpattern of length (oe) 


(a) Prove that there exists a k-superpattern of length (a) 


(b) Prove that there exists a k-superpattern of length [(n?/2)]. 


Can one pattern be Wilf-equivalent to infinitely many? That is, do 
there exist patterns q and qi, q2,--- so that Av,(q) = Avn(q1,q2,°--)? 
In order to exclude trivial answers, we require that the infinite sequence 
qi; 92,°°: consist of patterns of pairwise distinct sizes. 


Let q be a layered pattern that contains exactly two layers, and each of 
those layers is of length at least two. Prove that q-optimal permutations 
are all layered, and also have exactly two layers. 


Prove that for all constants c < 1, the inequality allperm(n) > en? holds 
if n is large enough. See Exercise 31 for the definition of allperm(n). 


Prove that g(2413) > 0.10472. 


Let n > 10. Prove that allperm(n) < n? — 2n+ 3. Note that this result 
is barely sharper than the result of Exercise 32. 


Let n > 7. Prove that allperm(n) < n?—-4+#. This bound is sharper 
than that of the preceding Problem Plus if n > 11. 


Prove that the upper bound given in Exercise 37 for spp(n) is not opti- 
mal if n > 6. 
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25. What is the length of the shortest permutation that contains all 2”~! 
layered permutations of length n? 


26. (a) Let p be a permutation that contains all g-avoiding permutations 
of length k, and is of minimal length among all permutations with 
that property. Is it true that p avoids q? 


(b) Consider the set S$ of all permutations of minimal length that con- 
tain all g-avoiding permutations of length k. Is it true that at least 
one element of S is g-avoiding? 


27. Improve the lower bound that was given for sp(k) in Exercise 22, part 


(a). 


8 i i i i 
Solutions to Problems Plus 
1. This result is due to John Noonan [255], who proved that 


f(m) = $123,1(m) = : ( sa ) 


n\n+3 


Doron Zeilberger [339] found a simpler proof of this formula. The P- 
recursive property then follows as 


i) re ee (2n)! (n + 2)!(n — 4)! 2n(2n — 1) 


f(n—1) n (n+3)\(n—3)! (Qn—2)!! = (n+3)(n—3)’ 
2. (a) Recall that Theorem 4.22 says that 


(k — 1)?" 


where A, is a constant given by a multiple integral. In particular, 
if k > 2 is even, then the denominator of the right-hand side is a 
polynomial of the form n?, where n is a positive integer. However, 
it can be proved (see for instance [168], Theorem D, page 293) that 
the coefficients of an algebraic power series cannot be of this form. 


— 
=z 


In fact, if f(z) = 0,30 fnz” is an algebraic power series and ay ~ 
cn4a", where a and c are non-zero constants, and d is a negative 
real number, then d = s +. 0.5 for some integer s. (See the solution 
of Exercise 6.3 in [297].) Therefore, the generating function of the 
sequence SAv,,(1234---k) is not algebraic if k is an even number 


that is larger than 2. 
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. This injection can be found in [83], where it is built up from an idea of 


Doron Zeilberger. 


. This result is proved in [83]. 


. It follows from the previous Problem Plus that for all n, the inequality 


Ss216r,1(n) < Avn+42(231 6 r) holds. It is easy to see that the two 
sequences have the same growth rates, so their generating functions 
have the same convergence radius R. 


By Corollary 5.21, the generating function of the sequence Avy+2(2316 
r) has a finite value at R; therefore, so does that of the sequence 
S3216r,1(1). However, all singular points of a rational function are poles, 
and power series go to infinity at poles. 


. This result is proved in [83]. The main idea is that 


Avn(k pea 21) < Sk(k—1)---21,1(7) < AvVn+o(k sas 21), 


and the two sequences squeezing S)(4—1)...21,1() have non-algebraic gen- 
erating functions, as shown in Problem Plus 2. 


. This formula was first conjectured in [256]. It was proved by Toufik 


Mansour and Alek Vainshtein [249], who found a general method to 
compute the ordinary generating function of the numbers S132,,(), for 
r<6. 


. This formula was also conjectured in [256]. It was proved by Marcus 


Fulmek [184], who used a lattice path approach. 


. This result is due to the present author and can be found in [50]. The 


proof is very similar to the proof of $j32,-(n) being polynomially recur- 
sive. The generating function is algebraic, and has only one quadratic 
irrationality, namely V1 — 4z. 


Let p(n) be the n-permutation 1n2n—1---, with p(0) =9, p(1) = 1, 
p(2) = 12, p(3) = 132, p(4) = 1423, p(5) = 15243, and so on. Then the 
value of f(p(n)) for these permutations is 1, 2, 3, 5, 8, 13. This suggests 
that f(p(n + 2)) = f(p(n + 1)) + f(p(n)). 


We are now going to prove this claim. Note that all patterns contained 
in p(n + 2) start either in their minimal or in their maximal entries. 


(a) The patterns that start in their maximal entries. These patterns 
are clearly contained in p(n+2) even if the initial entry 1 of p(n+2) 
is omitted. Then the entry n+ 2 can play the role of the maximal 
entries of these patterns, showing that there are pat(p(n)) such 
patterns. (Removing 1 and n+ 2 from p(n + 2), we get p(n).) 
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11. 


12. 


13. 


14. 


(b) The patterns that start in their minimal entry. The entry 1 of 
p(n + 2) can play the role of the minimal entry for these patterns. 
Then these patterns can continue p(n + 1) different ways. Indeed, 
removing 1 from p(n + 2), we get a permutation with f(p(n + 1)) 
different patterns as the obtained permutation is the complement 
of p(n + 1). 


Therefore, the sequence f(p(n)) is at least as large as the Fibonacci 
sequence, proving our claim. This construction is due to Herb Wilf. 


This result is due to Micah Coleman [122]. The proof is as follows. Let 
pr be the n-permutation 


Pe=k2k... k2 (k—-1) ... (k?-1) ...... 1 (k+1) ... (k?—-k+1). 


For example, 
p3 =369 258 147. 


In other words, pz consists of k segments, each of which consist of k 
entries that are congruent to one another modulo k. 


We claim that pz has at least 2(k-1)” different patterns. Indeed, let P, 
be the set of subsequences of p; that 


(a) Contain all k entries divisible by k, and 
(b) Contain all entries of [A]. 


In other words, subsequences in P; must contain the whole first segment, 
and the first element of each segment. This means that these patterns 
must contain these 24 — 1 entries, and are free to contain or not to 
contain the remaining (k — 1)? entries. Therefore, Pp consists of 24°—))” 
subsequences. The reader is invited to verify that these subsequences 
are all different as patterns. 


The first construction proving this fact was given by four authors in [7], 
who showed that for all positive integers n, the inequality 


pat(n) > 2” (1 = oyna vr?) 


holds. 


This result is due to Alison Miller [252]. This is the best currently known 
estimate for pat(n). The extra strength of Miller’s method is that she 
obtains estimates on the number of patterns that have to be contained 
multiple times in a permutation. 


We claim that 519472683 is a 4-superpattern. To verify this, note that 
the last five entries form a 3-superpattern. The entries 1 and 9 are on 


254 Combinatorics of Permutations, Third Edition 
7|@)|9 
4|S|@ 
1|@)| 3 

FIGURE 5.2 


The chosen squares lead to the subword 48526, then to p(S) = 25314. 


15. 


16. 


ve 


the left of that 3-superpattern, so we certainly have all patterns that 
start with 1 or 4. One then verifies that the remaining 12 patterns are 
also present. This construction is due to Rebecca Smith. Note that 
computer data proves that there is no 4-superpattern of length 8. 


This result was proved in [166]. The authors proved the upper bound 
constructively as follows. Let us consider an m x n chess board, in 
which, like in real chess, the bottom right corner square is white. Let us 
pick a subset S of t squares on this chess board. We will associate a t- 
permutation p(S) to S as follows. From left to right, write the numbers 
1 through n in the first (bottom) row in increasing order, the numbers 
n+1,---,2n, in the second row in increasing order, and so on. Then 
read the entries that belong to S column-by-column, starting with the 
leftmost column, and going down in each column. See Figure 5.2 for an 
example. 


This will result in a pattern of length t, which then defines a t-permutation 
by relabeling. The authors then take the k x |3n/2]| chess board, and 
choose S$ to be the set of all white squares. They prove that the resulting 
permutation is in fact a k-superpattern. 


This result was proved in [166]. Let S = Tj, be the set of squares weakly 
below the diagonal of the k x k rectangular chess board. It is then 
proved that the permutation p(k) defined by these squares (in the sense 
of the solution of the previous Problem Plus) contains q or q~! for each 
pattern q of length k. 


(a) This result is due to Alison Miller [252]. Let Z;, be the word of 
length k(k + 1)/2 that starts with the increasing sequence a, = 
13...(2|k/2] +1), continues with the decreasing sequence (2|(&+ 
1)/2])...4 2, then continues with alternating copies of a, and bx, 
until [k/2] copies of a, and |k/2| copies of b; are used. For in- 
stance, 

Zs = 135642135642135. 


We call Z, a zigzag word. The author then proves that either 
Zy, or Zp +1 (the word obtained from Z, by adding 1 to each 
entry) contains all permutations of length k. She then shows that 
if p = pip2-+* Pk(k-+1)/2 18 an n-permutation so that p; < p; if the 
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18. 


19. 


20. 


21. 


22. 


ith letter of Z, is smaller than the jth digit of Z,, then p is a 
k-superpattern. 


(b) This result is due to Michael Engen and Vincent Vatter [164]. The 
proof is similar to that seen in part (a), but the definition of the 
zigzag words is more careful. If n is even, then there is an increasing 
run of the first n/2 odd integers, followed by a decreasing run of 
n/2 integers, and so on. For instance, we get 


26 = 185643135642135642. 


If n is odd, then the odd runs are of length (n + 1)/2, and there 
are (n+ 1)/2 of them, while the even runs are of length (n — 1)/2, 
and there are (n — 1)/2 of them. For instance, 


25 = 1354213542135. 
Just as in part (a), breaking the ties in z, results in a k-superpattern. 
Yes. It is proved in [22] that Av,(1342) = Avn(q2,q3,--:), where 
dm = 2 2m—-—14163---2m 2m —3, 
for m > 2. 


This result was proved in two parts. First, it was shown in [268] that 
if p is a layered q-optimal permutation, then p has only two layers. 
The somewhat lengthy argument shows that if there exists a layered 
n-permutation with s+ 1 layers containing k copies of q, then a layered 
n-permutation containing more than k copies but consisting of only s 
layers also exists, as long as s > 2.) Then, it was shown in [4] that 
if q is as specified in the exercise, then all q-optimal permutations are 
layered. Note that the latter result does not need the requirement that 
q has only two layers, just that all layers of q are of length at least two. 


It is proved in [227] that for any real number a € (1.75, 2), there exists 
a constant cq so that allperm(n) > n? — can, for all n > 1. The proof 
is quite complicated, though it can be called an enhanced version of the 
argument given in the solution of Exercise 33. 


This strong result was proved by Cathleen Battiste Presutti and Walter 
Stromquist [266] by a high-powered argument. The authors conjecture 
that their lower bound, given by an integral and computed to 20 deci- 
mals, is actually equal to g(2413). In [265], a simpler argument is used 
to obtain the slightly weaker result that g(2413) = 0.10425. 


This result is due to Eugen Zalinescu [335]. While it is numerically a 
small improvement, it is remarkable because it ended a 35-year long 
dormant period in the history of this problem. 
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23. This result is due to Sasa Radomirovié [270], who improved on Zalinescu’s 
result just one year later. 


24. For n = 6, Exercise 37 provides the upper bound spp(6) > 873, but 
Robin Houston [217] found a 6-superpermutation of length 872. It then 
follows from the recursive result given in the solution of Exercise 37 that 
spp(n) < op_ kl ifn > 6. 


25. Let a(n) be the length of the shortest such permutation. Then it is 
proved in [8] that 


a(n) =n+ oo a +a(n—k-—1)}. 


It has been shown by Knuth ([230], Section 5.3.1, equation (3)) that this 
implies that 


a(n) = (n+1)flogy(n + 1)] — 2lee2(™F D1 4 1, 


26. (a) No. The following counterexample is given in [8]. Let & = 5, let 
q = 231, and let 


p=15119328476 10. 


(b) No. The counterexample of part (a) is a counterexample for this 
statement as well. 


27. It is proved in [117] that if n < 1.000076k?/e?, then every permutation 
of length n contains only o(k!) different patterns. So for k large enough, 
sp(k) > 1.000076k?/e?. 


6 


Mean and Insensitive. Random 
Permutations. 


6.1 Probabilistic Viewpoint 


In the previous chapters we have enumerated permutations according to var- 
ious statistics. A similar line of research is to choose an n-permutation uni- 
formly p at random, and compute the probability of the event that p has a 
given property A. Throughout this chapter, when we say that we select an 
n-permutation at random, we mean that each of the n! permutations of length 
n is chosen with probability 1/n!. 

Theoretically speaking, this is not a totally new approach. Indeed, the 
probability of success (that is, the event that p has property A) is defined as 
the number of favorable outcomes of our random choice divided by the number 
of all outcomes. In other words, this is the number of n-permutations having 
property A divided by the number of all n-permutations, which is, of course, 
n!. Therefore, the task of computing the probability that p has property A is 
reduced to the task of enumerating n-permutations that have property p. 

The formal definition of discrete probability that we are going to use in this 
book is as follows. 


DEFINITION 6.1 Let © be a finite set of outcomes of some sequence 
of trials, so that all of these outcomes are equally likely. Let B CQ. Then we 
call Q asample space, and we call B an event. The ratio 


_ |B 


P(B) = 


is called the probability of B. 


Methods borrowed from probability theory are often extremely useful for 
the enumerative combinatorialist. In this chapter, we will review some of 
these methods. 

In the most direct examples, no specific knowledge of Probability Theory 
is needed. We just need to look at our object from a probabilistic angle. Let 
us start with some classic examples. 
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Example 6.2 
Let 2 and j be two distinct elements of [n], and let p be a randomly chosen 
n-permutation. Then the probability that 7 and 7 belong to the same cycle of 
pisi/2. O 


SOLUTION For obvious reasons, the probability in question does not 
depend on the values of 2 and j, so we might as well assume that 1 = n — 1 
and j =n. 

The crucial idea is that we can use the Transition Lemma (Lemma 3.39). 
Indeed, entries n — 1 and n are in the same cycle of p if, when p is written in 
canonical cycle notation and the parentheses are omitted, n precedes n — 1. 
This obviously happens in half of all n-permutations, proving our claim. 


This result can be illustrated as follows. I go to a movie theater where each 
customer is given a ticket to a specific seat. Careless moviegoers, however, do 
not respect the assigned seating, and just take a seat at random. When I go 
to my seat, it can be free, or it can be taken. If it is free, fine. If somebody 
else has already taken it, I ask her to go to her own seat. She does that, and if 
that seat is free, then everybody is happy; if not, then she will ask the person 
illegally taking her seat to go to his or her assigned seat. This procedure 
continues until the person just chased from his illegally taken seat finds his 
assigned seat empty. Then the probability that a randomly selected person 
will have to move during the procedure (that is, that he is in my cycle) is one 
half. 

The previous example discussed a “yes-or-no” event, that is, two entries 
were either contained in the same cycle, or they were not. Our next observa- 
tion is more refined. 


Example 6.3 

Let i,k € [n], and let p be a randomly selected n-permutation. Then the 
probability that the entry i is part of a k-cycle of p is 1/n; in particular, it is 
independent of k. 


This fact is a bit surprising. After all, one could think that it is easier to 
be contained in a large cycle than in a small cycle as there is “more space” in 
a large cycle. 


SOLUTION We would like to use the Transition Lemma again. For 
obvious reasons related to symmetry, the choice of the entry 7 is insignificant, 
so we might as well assume that i = n. The bijection f of the Transition 
Lemma maps the set of n-permutations in which n is part of a k-cycle into 
the set of n-permutations in which py41-~ =n. The latter is clearly a set of 
size (n — 1)!, and our claim is proved. 
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1|2|5|7 |10 


FIGURE 6.1 
A Standard Young Tableau on ten boxes. 


In the context of the explanation following the previous example, this means 
that it is just as likely that 7 people in the movie theater have to move as it 
is that 7 people have to move, no matter what i and 7 are. 


6.1.1 Standard Young Tableaux 


Our simple examples in the previous subsection might have given the impres- 
sion that a proof using elementary probability arguments is necessarily simple. 
This is far from the truth. We illustrate this by presenting two classic proofs 
given by Curtis Greene, Albert Nijenhaus, and Herbert Wilf [201]. 

A Standard Young Tableau is a Ferrers shape on n boxes in which each 
box contains one of the elements of [n] so that all boxes contain different 
numbers, and the rows and columns increase going down and going to the 
right. Standard Young Tableaux have been around for more than one hundred 
years by now, being first defined by the Reverend Young in a series of papers 
starting with [334] at the beginning of the twentieth century. Figure 6.1 shows 
an example. 

Given this definition, a real enumerative combinatorialist will certainly not 
waste any time before asking how many Standard Young Tableaux exist on 
a given Ferrers shape, or more generally, on all Ferrers shapes consisting of n 
boxes. Fortunately, we can answer both of these questions, which is remark- 
able as we could not tell how many Ferrers shapes (that is, partitions of the 
integer n) exist on n boxes. Standard Young Tableaux, or, in what follows, 
SYT, are very closely linked to permutations. We will see in the next chapter 
that the entries of an SYT determine a certain (restricted) permutation, in- 
deed, an involution, of [n], and that there are numerous beautiful connections 
between SYT on n boxes, and the set of n! permutations of length n. All the 
above motivate us to discuss SYT in this book. 


6.1.1.1 Hooklength Formula 


DEFINITION 6.4 _ Let F be a Ferrers shape, and let b be a box of F. 
Then the hook of b is the set Hy of boxes in F that are weakly on the right of 
b (but in the same row) or weakly below b (but in the same column). The size 
of Hy is called the hooklength of b, and is denoted by hy. Finally, the box b 
is called the peak of Hp. 
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7|6/4]2]1 
4|3 |1 
2/1 
FIGURE 6.2 
A hook and all the hooklengths of a Ferrers shape. 
1|2 |3 2 
4| 5|6 3| 5| 6 
hooklengths 
1/|2(5 1, 3/4 4|3 12 
3 [4/6 2| 5| 6 3|2 {1 
1,3 [5 
2| 4/6 
FIGURE 6.3 


The five SYT of shape F = 2 x 3 and the hooklengths of F’. 


See Figure 6.2 for the hook Hy of b and the hooklengths associated to each 
box of a Ferrers shape. 

The number of SYT of a given shape is given by the following classic theo- 
rem. 


THEOREM 6.5 
[Hooklength Formula] Let F be any Ferrers shape on n boxes. Then the 
number of Standard Young Tableaux of shape F' is equal to 


n! 
II, hy’ 


where the product is over all n boxes b of F. 


(6.1) 


Example 6.6 

Let F' be a 2 x 3 rectangle. Then there are five Standard Young tableaux of 
shape fF’, and the hooklengths of F' are, row-by-row, 4, 3, 2, 3, 2, 1. So the 
Hooklength formula is verified as aaa = a = 5. See Figure 6.3 for an 
illustration. 
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b 


FIGURE 6.4 
Events A, and A, are not independent. 


This is one of those theorems that looks unbelievable at first sight, then 
believable, and then unbelievable again until a proof is completed. Indeed, at 
first sight it is not even obvious that the expression in (6.1) is an integer. 

On second thought, we could argue like this. Let us write in the entries of 
[n] into F in some random order. The obtained tableau will be a Standard 
Young Tableau if and only if each hook has its largest entry in its peak. The 
probability of that happening for a given hook Hy is ai so if we can multiply 
these probabilities together, we are done. 

Unfortunately, this is too big of a leap in the general case. Let A, be 
the event that Hy, has its largest entry at its peak. While it is true that 
P{Ap] = ae it is in general not true that P[Ay™ A-] = P[Ap]-P[Ac]. In other 
words, the events Ay and A, are not independent; the occurrence of one can 
influence the occurrence of the other. To see this, consider the Ferrers shape 
shown in Figure 6.4, with the boxes b and c marked. In the unlikely case 
that the reader has not seen an introductory probability text on independent 
events, it is very easy to catch up by consulting such a text. As we only need 
discrete probability in this book, we recommend Discrete Probability by Hugh 
Gordon [198]. 

Clearly, P[Ap] = P[A-] = $ holds, but P[A,M A-] = 4 as that is the chance 
that the element x in the bottom right corner is the smallest of the three 
elements involved. This kind of dependence of the events A, will always be 
present, unless F itself is a hook. Therefore, in the general case, this argument 
will not work. And this is why our theorem looks so unbelievable again. What 
we have to prove is that when everything is taken into account, we do have 
P(Nver Ab] = IL, a after all. 


PROOF (of Theorem 6.5) We will define an algorithm that generates a 
random SYT on F. We will show that each Standard Young Tableau T of 
shape F’ has the same chance, namely 


Pir] = He (6.2) 


chance to be generated by our algorithm. As our algorithm always stops by 

producing an SYT of shape F’, these probabilities have to sum to 1, meaning 

that there must be In of them, proving that this is the number of SYT on 
b 

F. 
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FIGURE 6.5 
Random choices leading to the placement of n. 


The algorithm proceeds by first placing the entry n somewhere in F’, then 
placing the entry n — 1 somewhere else in F’, and so on, until all entries have 
been placed. 

The entry n is placed as follows. First, a box in F' denoted by q is chosen 
at random. (Here, and throughout this proof, all eligible boxes have the same 
chance to be chosen.) Then, g is moved to any of the other h, — 1 positions 
of the original hook H,. Call this new box q now. Then repeat the same for 
the new q, that is, move it to a different position within the new Hy, and 
call that box g. Continue this until g becomes an inner corner, that is, a box 
whose hook consists of one box only, namely q itself. When that happens, 
place the entry n into g. The process of moving n to its final destination that 
we described above is called a hook walk. 


Example 6.7 

Figure 6.5 shows a possible hook walk to the placement of n = 10 in our 
Ferrers shape. The first choice has 1/10 of probability, the second one has 
1/5, and the third one has probability 1/3. Therefore, the probability that 
this particular hook walk will be chosen when we place n is Ee 


Once n has been placed, temporarily remove the box containing n from F’ 
to get the Ferrers shape F\, and repeat the algorithm with n — 1 playing the 
role of n. Continue this until all elements of [n] are placed and an SYT is 
obtained. It is obvious that this algorithm always produces an SYT of shape 
F. 

We promised to prove that all SYT on F are obtained with the same proba- 
bility by this algorithm. We prove this statement by induction on n, the base 
case of n = 1 being trivially true. 

Let T be an SYT having shape F that contains the entry nm in box gq. 
Removing that box, we get the Standard Young Tableau T, that has shape 
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F,. It is then clear by the structure of our algorithm that 
PIT] = P[T:| Pla], (6.3) 


where P[q] is the probability that n gets placed into box q, while P[T] and 
P{T,] are the probabilities that the mentioned tableaux are obtained by our 


algorithm. 
As T; is an SYT on n— 1 boxes, by the induction hypothesis, we have 
Ther hy 
PIT)| = 1 6.4 


Comparing equations (6.3) and (6.4), we see that our main claim (6.2) will 
be proved if we can show that 


1 he 
Pil=—- [1 7 (6.5) 
6elig 


where L, is the set of boxes that are in the same line (row or column) as gq, but 
are different from q. Indeed, if d ¢ L,, then the removal of q does not affect 
the hooklength hg, so hg will be part of both the numerator and denominator, 
and will therefore cancel. Otherwise, the removal of g will decrease hg by one, 
explaining the remaining terms of the last equation. Recall that q is a corner, 
so for all boxes c € Lg, the inequality h, > 1 holds, so (6.5) will never contain 
a division by 0. 

In order to better understand what P{q] is, we decompose it as a sum of 
several summands corresponding to the various sequences leading to q. If 
we have a sequence of random choices leading to the placement of n into 
q = (a,y), then we denote by J the set of rows (not including x) and by J 
the set of columns (not including y) that contained the moving box gq at some 
point during the sequence. Then J and J are called the horizontal and vertical 
projection of that sequence. 

Let Pr,z[x,y] be the probability that a random sequence of choices leads to 
q = (a, y) and has horizontal projection J and vertical projection J. 


Example 6.8 

Let F' be as shown in Figure 6.6, and let g = (2,3). If J = {1} and J = {2}, 
then there are two possible sequences ending in q; one is (1,2) — (1,3) > 
(2,3), and the other is (1,2) — (2,2) — (2,3). We have seen in Example 6.7 
that the first sequence has a probability to be selected, whereas the second 


one has 7: $+ 5 = ag probability to be selected. This yields 
1 1 1 
P. = —— —— | 
1al@ 9 = 755 * Too = 60 
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q 
FIGURE 6.6 
The projections J and J. 
It is now obvious that 
=) Psiea: (6.6) 
I,J 


It is time that we proved (6.5). Note that we can rearrange (6.5) as follows. 


roi=2 TE (tgs) =4 TL (+2) TL (2) 


c€Lyg 1<i<a-1 1<j<y-1 
(6.7) 


where we split the set Lg into two parts, those boxes that are in the same row 
as q and those boxes that are in the same column as q. So a; is the hooklength 
of the box (i, y) decreased by one, and 0; is the hooklength of the box (z, 7) 
decreased by one. 

We are going to prove (6.7) by showing that each expansion term of the 
last expression is equal to a suitably selected Pr ;{x,y]. Then (6.7) will im- 
mediately follow by (6.6). 

In fact, we claim that the following holds. 


LEMMA 6.9 
Let x and y be the coordinates of an inner corner q, and let I C [x — 1] and 
JC [y—1]. Then the equality 


1 1 1 
Pr, z(x,y] = 7 I - I a 
* % - bi] 

tel jEed 


holds. 


Example 6.10 
Let F' be the same shape as in Example 6.8, let « = 2, and = = 3, and let 
I = {1}, and J = {2}. Then we need to compute the product 2 a El : E: As 
we said after equation (6.7), a; is the hooklength of the box a 3) decreased 
by one, that is, aj = 4—1 = 3. Similarly, bg is the hooklength of the box (2, 2) 
decreased by one, that is, b: = 3-—1= 2. This yields — . 3 . 4 = a: This is 
what we expected as in Example 6.8 we have computed that Pr,j[v,y] = a. 


Tl 60 
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PROOF = (of Lemma6.9) In this proof, it will be advantageous to denote a 
box by its coordinates (a,b) rather than just by one letter, and consequently, 
hooks and hooklengths will also be denoted by Ha,» and hap. 

Let us assume first that J is empty. That means that the hook walk leading 
to q started in row y, and therefore its horizontal projection J completely 
determines it. So in this case, Py,;[x,y] is equal to the probability of that 
single hook walk, and that is obviously +TI ier r= We argue analogously if I 
is empty. 

Let us assume now that both J and J are nonempty. In this case we prove 
our lemma by induction on |J| + |J|. The initial case is when J = {7} and 
J = {j}, and this case is easy to verify. 

To prove the inductive step, note that if a hook walk has I = {71, %2,--- , tx} 
as its horizontal projection, and J = {j1, j2,--+ , jx} for its vertical projection, 
then it must start at (i1, 71). Then it must continue either to (i2, 71) or to 
(41,j2). The hook walks from either of these two points to (a,y) have a 
truncated horizontal or vertical projection, that is, /’ = I — {i} or J’ = 
J — {j,}; therefore, the induction hypothesis applies. If h is the hooklength 
of the hook H of (¢1, 71), then it is immediate from the definitions that 


1 
Pr, 7|2, y] _ Ao Pr lel + Pr,3'|2, y)). 


However, by the induction hypothesis this implies 


1 1 ay b; 
Pralewl = = (| pt 
h-1l n Hier ay Hoes bj Hier 7) [jeu 8 
ai, + bj, : 1 
h-1 MVier %* Wjes bs 


Finally, note that the first fraction of the last row is equal to one. Indeed, 
by the definition of the a; and b;, we have 


Qi, + b;, =hi,y —l+hey, —l=h—-1, 


as |Hi, U Hej,| = |H| = h. Note that |Hi, 9 Hz,;,| = 1, because that 
intersection consists of the box q (see Figure 6.7). So the lemma is proved. 


This proves (6.7) and therefore the equivalent equality (6.5), which in turn 
implies our main claim (6.2). | 


6.1.1.2 Frobenius Formula 


The other spectacular result on the enumeration of Standard Young Tableaux 
is the Frobenius formula, which also dates back to the beginning of the twen- 
tieth century [181], [182]. 
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FIGURE 6.7 
The gray hooks intersect in one box. 


FIGURE 6.8 
The Ferrers shapes on five boxes and their f’-value. 


THEOREM 6.11 
[The Frobenius formula] For a Ferrers shape F, let f* denote the number of 
Standard Young Tableauz that have shape F. Then for any positive integer n, 


we have 
> FP =n, (6.8) 


|Fl=n 


where the sum on the left-hand side is taken over all Ferrers shapes on n 
boxes. 


Example 6.12 

Let n = 4. Then there are five SYT on n boxes, and the values of f” for 
these SYT are 1, 1, 2, 3, 3 as shown in Figure 6.8. So the Frobenius formula 
is verified as 17 + 1? + 2? + 3? + 3? = 24. 


The Frobenius formula is even more surprising than the hooklength formula. 
It has a nice and simple bijective proof, which we will cover in Section 7.1. It 
also has various algebraic proofs. Here we will show a probabilistic proof that 
is again due to Greene, Nijenhaus, and Wilf [202]. 
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The outline of the proof will be as follows. We will define a random pro- 
cedure that produces a random SYT on n boxes. Note that we do not know 
in advance what the shape of the obtained SYT will be. Then we prove 
that our procedure produces any given SYT of shape F’ with probability 
f*/(n!). Therefore, the total probability that we get a tableaux of shape F 

oi 


is f* . i = Gy). proving that 


FY2 
alae 
FP 


as claimed. 
We will need the notion of conditional probabilities. 


DEFINITION 6.13 Let A and B be two events so that P{B| > 0. Then 
we define 
P{AN BI 

P[B| 


and we call P|A|B] the probability of A given B. 


P[A|B] = 


In other words, P[A|B] is the probability of the occurrence of A if we assume 
that B occurs. As we mentioned before, A and B are called independent if 
P{A|B] = P[A], that is, if the occurrence of B does not make the occurrence 
of A any more likely or any less likely. 

One basic and well-known application of conditional probabilities is Bayes’ 
Theorem, which is also called the law of total probability. It can be found in 
any introductory probability textbook, such as [198]. It states the following. 


THEOREM 6.14 
Let AC Q be an event, and let X1, X2,---,Xm be events in Q so that the 
X;, are pairwise disjoint, and X,U X2U---UXm =. Then the equality 


P(A) = S © P(A|X;)P(X;) 


i=l 


holds. 


In the proof of the hooklength formula, we considered various probabili- 
ties that a hook walk ends in a particular box. Now we will be looking at 
conditional probabilities P[ab|zy] that a hook walk ends in a given box (a, b) 
provided that it went through another given box (z, y). 

We want to prove some general facts, preferably explicit formulae, about 
P{ab|zy]. The first step is the following proposition that provides these for- 
mulae for hook walks that lie within one row or column. 
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PROPOSITION 6.15 
Let x > a and let y > b. Then 


hi 
Playlay] = ane (6.9) 
a<i<a VY 
and similarly, 
hey.; 
Plry|ad] = |] a (6.10) 
bSj<y 74 


PROOF _ Ifa hook walk passes through (a, b) and goes to (x, y), then its 
first stop after (a,b) can be any of the other hay — 1 boxes of Ha». After that 
stop, the walk can proceed to (#,y) in many ways. Therefore, by Theorem 
6.14, 


P{xy|ab] = 


ha a >. Pleylid]+ D7 Pleulad | - (6.11) 


aci<a b<j<y 


If b= y, then the second sum is empty, and we get, after rearrangement, 


(hay — 1)Plzylay] = D2 Pleyliy). (6.12) 


a<i<a 


In order to get rid of the factor ha, — 1, we apply the following clever trick 
[202]. We take (6.12) for a+ 1, instead of a. Then we get 


(Ra+1,y —1)Pleyl(at+1jyl= D> Pleyliyl, (6.13) 


atl<i<a 
Subtracting (6.13) from (6.12) we get 


(Ray — 1)Playlay] — hatiy — YPlayl(a + 1y] = Playl(a+ Dy, 


hat, 
Plxylay] = 7 Pleyl(a + yl. 
ay 


We have obtained a formula for P[xy|ay] that is the product of P[ay|(a+ 1)y] 
and another term, namely aoe. Applying the same procedure again, this 
time for Plxy|(a + 1)y], then Play|(a + 2)y], and so on, we will be left with 
the product of fractions, and get (6.9). 


The proof of (6.10) is analogous. 


A direct consequence of this Proposition is the following, surprisingly com- 
pact expression for P[xy|ab]. 
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COROLLARY 6.16 
Letx >a andy>=>b. Then 


Plxylab] = Plxylay] - Piay|xb). (6.14) 


PROOF _ Let F be the rectangular Ferrers shape with northwestern and 
southeastern corners (1,1) and (x,y), and let S C F be the subset of boxes 
(a,b) in F for which (6.14) is not proved yet. So, at the beginning, S = 
F — (x,y). Let (a,b) be an inner corner of S. We will now prove that (6.14) 
must hold for (a,b). 

Let us look at (6.11) again. In the first summation, we have a < i < 2; 
therefore, the boxes (7, b) are below the box (a, 6), that is, they are in F—S 
(since (a,b) is an inner corner of S), and so we can apply (6.14) to replace the 
terms P[xy|ib] by the terms Pixy|iy]P[xy|xb]. This leads to 


S> Pleylit] = So Pleyliy|P[zy|xd] = Pleyleb] S> Pleyliy] (6.15) 


a<i<a a<i<a a<i<a 


Plxy|xb|Playlay|(hay — 1) (6.16) 


where in the last step we simply used (6.12). 
If we transform the second summation in (6.11), we get that 


S> Playlag] = Plrylay|Plxy|ab|(he,» — 1)- (6.17) 
b<jsy 


Finally, we replace the two sums in (6.11) by the expressions we just computed 
in (6.15) and (6.17) to get 


P{xy\ay|P[xy|xb] (hay — 1+ hep — 1) 


Plzylab] = at 


= P[xylay]Plaxy|xd), 
as it is clear that hay + he» — 1 = ha». See Figure 6.7 for an illustration of 
this. 

Now that we proved our statement for (a,b), we can remove (a,b) from S$ 
to get 5”, and choose an inner corner of the new shape F' — S’. We can then 
repeat the whole procedure until (6.14) is proved for all boxes of F’. 


In order to continue our proof of the Frobenius formula, we need some new 
notions. Let F’ be the Ferrers shape of a partition \ = (a1, @2,--- , ax). Recall 
that these parts are written in non-increasing order. Choose p and q so that 
p > k and q > a. Then the (p,q)-complementary shape Fy, is the set of 
boxes that are in the rectangle spanned by (1,1) and (p,q), but not in F. 


Example 6.17 
Let F' be the Ferrers shape of (5, 4,3), and let (p,q) = (4,6). Then F' is shown 
in Figure 6.9, and F;,,, is represented by the shaded boxes. 


270 Combinatorics of Permutations, Third Edition 


FIGURE 6.9 
A shape and its complementary shape. 


Note that the outer corners of F’, that is, the boxes that we can add to F 
to get another Ferrers shape, are the same as the inner corners of FY ,, that 
is, elements that can be removed from Fy ,. For the rest of this proof, if an 
outer corner of FY, is denoted by K, and the box immediately on its left is 
an inner corner of F', then that box will be denoted by K. 

We define a special complementary hook walk in the complementary shape 
Fs, a8 a hook walk that starts at (p,q) (hence “special” ), and uses steps north 
and west (hence “complementary” ). These walks can stop at any of the inner 
corners of Fy .. 

Finally, we define the distance between two boxes in the obvious way, that 
is 

d((x,y),(v,2)) = |e -v| + ly al. 

In what follows, we will be interested in computing the probability that 
a special complementary hook walk stops in a given inner corner of Fy 4. 
The alert reader probably suspects that we are interested in this because our 
random procedure that will produce each SYT with the correct probability 
will be defined recursively, by ensuring that it puts the maximal element in a 
certain corner. The alert reader is right, of course. 

The following Lemma shows that the choice of p and q is not that crucial. 


LEMMA 6.18 

Let X = (a1, 42,-++ ,a@~) be a partition of n corresponding to the Ferrers 
shape F, and let p > k and q > ay. Then the probability that a special 
complementary hook walk in Fy, will end at inner corner K of Fog ts 


P(K) = Hate) (6.18) 


7 Hedk, Ry 


where R ranges over all inner corners of Fy 


g, and R ranges over all outer 
corners of F. 


Note that this means in particular that the probability P[K] is independent 
of the choice of p and gq. Indeed, the inner corners of FY’, are determined by 
the outer corners of F’, and so neither the numerator nor the denominator of 
the right-hand side of (6.18) depends on p and gq. 
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FIGURE 6.10 
A generic term that does not cancel. 


PROOF _ First, we will prove our statement for a regular (not a comple- 
mentary) Ferrers shape. So instead of special complementary hook walks, we 
will be looking at hook walks starting at (1, 1). 

Let K = (x,y) be the corner in which we want our hook walk to finish. 
Then we need to compute the probability P[xy|11]. Applying (6.14), then 
(6.9) and (6.10), we get 


Pley|11] = Pleylly|Pleylel] = JT] 7-4 I] 7. 


l<i<a 'Y l<j<y 74 


Taking a closer look at the fractions on the right-hand side, we see that many 
of them are equal to 1. For instance, look at the generic term p ot The 
horizontal parts of these hooks end in (a, y). The vertical parts are different: if 
and only if the 7th column of our shape ends in a corner R. It is in this case, 


and only in this case, that et # 1. See Figure 6.10 for an illustration. 
In this case, the denominator is equal to d(K, R), whereas the numerator is 
equal to d(K , R), where R is the box just below the end of column j + 1. 

If we change our shape to a complementary shape, our hook walk to a special 
complementary walk, and K to K, we get the statement of the lemma. 


Example 6.19 
Considering the Ferrers shape in Figure 6.10, let A = (3,4), and let us look 


at the term 7 43.2 The value of this term is 3/6. On the other hand, we have 


d(K, R) =6 atid atk, R) = 3, verifying our argument. U 


The previous lemma provided a formula for P[K], but it was a rather com- 
plicated one. The following lemma will bring that formula closer to what we 
need. 


LEMMA 6.20 
Let F be a Ferrers shape on m bozes, and let F’ be a shape obtained from F 
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ee ie 


FIGURE 6.11 
How corners cause non-canceling terms. 


by adding an outer corner K. Then 


a Ee 
PR = nF 


PROOF We can express both f” ‘and fF by the hooklength formula. 
After that, m! cancels, and so do all hooklengths that belong to hooks not 
being in the same row or column as K. 

What happens with the remaining terms? Let K = (x,y). Looking at the 
row of K, we see that the hook H,,; of F is as long as she hook Hf, ji of F", 
unless the vertical parts of these hooks are different, that is, vines tees isa 
corner R at the end of column j of F’. In that case, instead of a cancellation, we 
geta d(K, R)/d(K, R) factor. An analogous argument applies for the column 
of K. However, by Lemma 6.18, the product of these terms d(K, R)/d(K, R) 


is precisely P[K], proving our claim. 


Example 6.21 
Let F be the Ferrers shape of the partition (5,4,1), and let K = (2,5). The 
reader is invited to verify using Figure 6.11 that the hooks of F and F” have 
the same length if the peak of the hook is not in the second row and not in 
the fifth column. The reader is also invited to verify cancellations involving 
hyj of F and hi, 54, of F”, provided that column j of F does not end in a 
corner. 

If 7 = 1, then column j of F' ends in corner R. Then we have hg, = 5 
in F and hy, = 4 in F’. On the other hand, we have d(K,R) = 5, and 


d(K, R) = 4, verifying our argument. 


Now we are ready to describe the random procedure that will produce each 
SYT of shape F with probability f”/n!, for all shapes F on n boxes. 

Choose p and q so that they are larger than n. We construct a series 
of SYT 7o,7%,---,Z, so that T; has i boxes as follows. Obviously, To is 
the empty Erle For brevity, let 7; = = , the complementary shape of 
T; in the rectangle spanned by (p,q). For i — 1, we get 7; from T;_, by 
inserting the entry 2 to one of the outer corners of Th. The crucial question 
is which outer corner? This corner is chosen by taking the endpoint of a 
special complementary walk in Z;_}. 
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THEOREM 6.22 
Let T be a Standard Young Tableau on n boxes having shape F. Then the 
random procedure defined in the previous paragraph produces T with probability 


fF /nl. 


PROOF We prove our statement by induction on n, the initial case being 
trivial. Let T* be the SYT that we obtain from T by omitting n, and let F'« 
be its shape. Then, by our induction hypothesis, the probability that our 
algorithm constructs T’ is P[T'*] = f**/(n—1)!. When T is constructed, first 
Tx must be constructed (and the probability of this is f**/(n—1)!), and then 
a randomly chosen special complementary walk in the complementary shape 
of T* has to end in the corner K = F — Fx. The probability of the latter is, 
by Lemma 6.20, P[K] = f*/(n- f**). Therefore, we have 

n> Facile fe f? 
PT] = P[T*|P|K] = Gai! : i at 


completing the proof. | 


As we have explained after Example 6.12, Theorem 6.22 immediately im- 
plies Theorem 6.11. 


“8!  —=E ee 


6.2 Expectation 


We need a little bit more machinery in order to use stronger probabilistic 
tools. First of all, we formalize the common sense notion of probability we 
used in the previous section. 

A random variable is a function X :Q— R that associates numbers to the 
elements of our sample space. Most of the random variables we are going to 
work with will have a finite range, that is, the set of values they take will be 
finite. The sum and product of two random variables, and a constant multiple 
of a variable, is defined in the way that is usual for ordinary functions. As 
the reader has probably noticed, this book is mostly devoted to permutations; 
therefore, our sample space 2 will most often be S;,, and the random variable 
X will most often be some permutation statistic. Two random variables X 
and Y are called independent if for all 7 and j, the equality 


PIX =i,Y =j])=P[X =i]- PY =3] 


holds. 
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One of the most important statistics of a random variable is its expectation, 
defined below. 


DEFINITION 6.23 Let X :Q—> R be a random variable that has a 
finite range S. Then the number 


E(X) =)0i- P(X =) 


ies 
is called the expectation of X on Q. 


mb 99 06. 


Other names for E(X) include “expected value,” “expected number,” “mean,” 
“average,” or “weighted average.” The latter is meant with the individual 
probabilities as weights. If there is no danger of confusion as to what the 
variable X is, its expectation E(X) is sometimes denoted by p. 

For certain random variables, the expectation is easy to compute directly. 
This is often the case for variables defined by symmetric permutation statis- 
tics. 


Example 6.24 

Recall that d(p) is the number of descents of the permutation p. Let n 
be a fixed positive integer, and let Z : S, — R be the random variable 
defined by Z(p) = d(p) +1. That is, Z counts the ascending runs of p. Then 
E(Z) = (n+1)/2. 


PROOF — Recall that A(n,7) denotes the number of n-permutations with 
i —1 descents. Directly from the definition of F(Z), we have 


E(Z) = ye a 


Noting that A(n,i) = A(n,n + 1 — 7), we see that 


Ct eer eee ee le 


and the proof follows after summation over 7. | 


6.2.1 Application: Finding the Maximum Element of a Se- 
quence 


The fact that X was symmetric played a crucial role in the above argument. 
Nevertheless, there is another interesting general phenomenon of which the 
above example was a special case. Recall from Chapter 1 that the Eulerian 
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numbers A(n,k) are not only symmetric, but also unimodal; therefore the 
sequence A(n, k) has either its one or two maxima in the middle. 

Recall that the Eulerian polynomials A,,(z) are defined by the equation 
An(z) = Sp-1 A(n, k)z®. Therefore, A’,(z) = op_, KA(n, k)z*-1, and 


n 
Al, (1) = S$ kA(n, k) = nl B(Z). 
k=1 

In other words, E(Z) = An) = sel Noting that we did not use anything 
specific about the Eulerian polynomials, we conclude that the expectation 
of a permutation statistic on S,, can be obtained by substituting 1 into the 
derivative of the relevant generating function, and dividing the result by n!. 

Let A(z) = 503, aiz’ be the ordinary generating function of some sequence 
@1,42,°** ,@m that enumerates n-permutations according to some statistic. 


The expression A’(1)/n! or se may ring a bell for the alert reader. Indeed, 


recalling Darroch’s theorem (Theorem 3.25), we remember that if A(z) has 
real zeros only, then its sequence of coefficients has at most two maximal 
elements, and these are at distance less than one from AM) So we have 


AQ) * 
proved the following corollary of Darroch’s theorem. 


COROLLARY 6.25 

Let s be a permutation statistic on S,, and let a; = |p € Sy: s(p) = ij. 
Let Z be the random variable corresponding to s. Assume that the ordinary 
generating function A(z) = S7¥_, ajz" has real zeros only. Then the sequence 
@1,92,°** , Gn has either one or two maximal elements, and they are at dis- 
tance less than one from E(Z). 


In other words, the expectation of Z does not only help us to understand 
the average behavior of Z, but also (if the real zeros condition holds) provides 
near-perfect information about the location of the maxima of Z. 


6.2.2 Linearity of Expectation 


One of the reasons for which the expectation of a variable is a very useful 
statistic is the following theorem. 


THEOREM 6.26 
Let X and Y be two random variables defined over the same finite sample 
space Q. Then F(X +Y)=E(X)+ E(Y). 


Note that we do not assume anything about X and Y other than the fact 
that they are defined over the same sample space. It does not matter how 
(if at all) they were related to each other; whether they are dependent or 
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independent variables. In other words, the mean of the sum of two variables 
is insensitive to dependency relations, in case you were wondering about the 
chapter title. 


PROOF Let 21, %2,°-: ,2%n be the values that X takes with a positive 
probability, and let y1,y2,--- ,Ym be the values that Y takes with a positive 
probability. Then it follows from the definition of expectations that 


E(X+Y)=S° So (ai + yj) P(X = 2i,Y = y;) 


i=1 j= 
=S0) eP(X =24Y =y,) 
i=1 j= 
+0 So yjP(X = 2i,Y =y;) 
i=1 j= 
= So aiP(X =2;)+ So uP@ = yj) 
i=l j=1 
= E(X)+ E(Y). 


We point out that if a is a positive constant, then it is easy to prove that 
E(aX) = aE(X). This fact, together with Theorem 6.26 is often referred to 
by saying that F is a linear operator. 

The application of Theorem 6.26 often involves the method of indicator 
random variables as in the following example. 


Example 6.27 
Let n > 2. The expected number of 2-cycles in a random n-permutation is 
1/2. 


SOLUTION — Let X:S,, — R be the random variable giving the number 
of 2-cycles of a permutation. Let i and 7 be two distinct elements of [n], and 
let X;,; : S, — R be the random variable defined by 


1if¢ and 7 form a 2-cycle in p, 
Xij(p) = 
0 otherwise. 


It is then easy to compute the expectation of X;,;. Indeed, 


t= 2)h 1 
onl n(n — 1) 
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The observation that makes the variable X;,; useful for us is that summing 
X;j,;(p) over all 2-element subsets (7,7) C [n] we get X(p). Therefore, by 
Theorem 6.26, we obtain 


B(X)=E| d) Xi) = DU B(Xi3) = (5) Ba) = 5, 
i,j)C[n 


(4,9) ln] (1,9) In] 
as the choice of 2 and j is clearly insignificant. | 


The variables X;,;, or in general, variables taking values 0 and 1 depending 
on the occurrence of an event, are called indicator random variables, and are 
often very useful. 

The following theorem is not a very subtle, but very general, tool in proving 
that it is unlikely that a variable is larger than a multiple of its expectation. 
(We will state the theorem in a special case that is relevant in our combina- 
torial applications, but a more general treatment is possible.) 


THEOREM 6.28 
[Markov’s Inequality] Let X be any nonnegative random variable that has a 
finite range, and let a >0. Then the inequality 


1 
P[X > ap] < — 
a 
holds. 


In Section 6.4 we are going to prove a very similar inequality (Theorem 
6.48), which is often called Chebyshev’s inequality. The ambitious reader is 
invited to wait for the proof of that inequality, then to try to prove Markov’s 
inequality in a similar manner. Then the reader can check the proof that we 
will provide as the solution of Exercise 13. 

Markov’s inequality is particularly useful when the expectation of a variable 
is very low compared to its maximum value. 


Example 6.29 
Let a > 1. Then the number of n-permutations with more than a- In(n + 1) 
cycles is less than n!/a. 


SOLUTION This is immediate if we recall that by Lemma 3.26, the 
average n-permutation has H(n) = )7j_, + < In(n + 1) cycles. 
Note that this example is particularly striking when n is a very large num- 


ber. Let n be large enough so that n is roughly equal to 10000 In(n + 1). Let 
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FIGURE 6.12 
The tree T(p) for p = 328796154. 


a = 100. Then the probability that a randomly selected n-permutation has 
more than 100In(n + 1), or in other words, more than roughly n/100 cycles 
is less than 1/100. Speaking in more general terms, it is very unlikely that 
a randomly selected n-permutation will have at least cn cycles, regardless of 
how small the positive constant c is. 


a 


6.3 Application: Rank in Decreasing Binary Trees 


Let p = pp2---pn be a permutation. Let us recall that the decreasing binary 
tree of p, which we denote by T(p), is defined as follows. The root of T(p) is 
a vertex labeled n, the largest entry of p. If a is the largest entry of p on the 
left of n, and 6 is the largest entry of p on the right of n, then the root will 
have two children, the left one will be labeled a, and the right one labeled b. 
If n is the first (resp. last) entry of p, then the root will have only one child, 
and that is a left (resp. right) child, and it will necessarily be labeled n — 1 as 
nm —1 must be the largest of all remaining elements. Define the rest of T(p) 
recursively, by taking T(p’) and T(p”), where p’ and p” are the substrings of 
p on the two sides of n, and affixing them to a and b. 

Note that T(p) is indeed a binary tree, that is, each vertex has 0, 1, or 2 
children. Also note that each child is a left child or a right child of its parent, 
even if that child is an only child. Given T(p), we can easily recover p by 
reading T according to the tree traversal method called in-order. In other 
words, first we read the left subtree of T(p), then the root, and then the right 
subtree of T(p). We read the subtrees according to this very same rule. See 
Figure 6.3 for an illustration. 

The following definitions can be applied to all varieties of rooted trees, even 
though we will only explore it for decreasing binary trees. First, in a rooted 
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tree, we say that a descending path is a path that starts at a vertex v, and 
then each vertex in the path is a descendant of the preceding vertex. In other 
words, descending paths never go backwards. 


DEFINITION 6.30 Ina rooted tree, we say that vertex v is of rank k if 
the shortest descending path from v to any leaf of the tree consists of k edges. 


In other words, if v is of rank k, then the edge-distance between v and the 
closest leaf is k. So leaves are of rank 0, neighbors of leaves are of rank 1, and 
so on. 

We are interested in the following question. Let us say that n goes to 
infinity. From the set of all vertices of all decreasing binary trees on n vertices, 
we select a vertex uniformly at random. So each vertex has 1/(n!n) chance 
to be selected. What is the probability that the selected vertex is of rank k? 


6.3.1 Two simple initial cases 


Let us first handle the case of k = 0. That is, we are interested in the 
probability that a randomly selected vertex is a leaf. In order to alleviate 
notation, let us assume that all permutations p in this discussion are of length 
n, and so T(p) has n vertices. 

Let z(p) denote the number of leaves in the tree T(p), and let E(z) denote 
the expectation of z(p) taken over all permutations p of length n. 


PROPOSITION 6.31 
For all integers n > 2, the equality E(z) = = holds. 


PROOF = Let p=p)p2:--py. Let 2 <i<n-—1. Then it is straightforward 
to prove, for instance by induction on n, that the vertex corresponding to p; is 
a leaf if and only if it is smaller than both of its neighbors, and that event has 
probability 1/3. On the other hand, if i € {1,n}, then p; is a leaf if and only 
if it is smaller than its only neighbor, an event of probability 1/2. Therefore, 
if we denote by z;(p) the indicator variable of the event that p; is a leaf, then 
by linearity of expectation we get 


n 


E(z) =5— E(%) = (n-2)- 


i=1 


ee 1 n+l 
3 I~ Sa 


It is perhaps a little bit surprising that the formula for entries of rank 1 is 
just as simple as the formula proved in Proposition 6.31. Let Y(p) denote the 
number of vertices of p that are of rank 1. 
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THEOREM 6.32 
Letn > 4. Then the equality E(Y) = Sent) holds. 


PROOF _ Let an,1 be the total number of vertices in all decreasing binary 
trees on n vertices that are of rank 1. Note that if n > 1, then each leaf 
must have a unique parent, and that parent must always be a vertex of rank 
1. However, some vertices of rank 1 are parents of two leaves. We will now 
determine the number d,, of such vertices, which will then yield a formula, 


2 PTS 


On = dn (6.20) 


for Gn,1, where n > 4. 

Let p; be a vertex that is of rank 1 and has two leaves as children. Let 
us assume for now that 3 <7 < n-—2 holds. Then p; is larger than both 
of its neighbors, and both of those neighbors p;_; and p,;+41 are leaves, so 
they are smaller than both of their neighbors, meaning that p;_; < pj—2, and 
Pi+1 < Pi+2. On the other hand p; must be smaller than both of its second 
neighbors; otherwise, its children could not be pj_; and p;+;. This means 
that out of the 120 possible permutations of the mentioned five entries, only 
four are possible, since p; must be the middle one in size, its neighbors must 
be the two smallest entries, and its second neighbors must be the two largest 
entries. So if Z;(p) is the indicator variable of the event that p; has two leaves 
as children (in which case p; is always of rank 1), then for i € [3,n — 2], 
we get E(Z;) = i = a If i = 1 ori = n, then p; cannot have two 
children. Finally, if 7 = 2 or 7 = n—1, then an analogous argument shows 
that E(Z;) = 4 = 4. Therefore, since Z = oS Z; denotes the number of 
vertices that have two leaves as children (and are therefore of rank 1), then 
by linearity of expectation we have 


(2) = > BZ) = tn-4)- at 


30 30 
Therefore, d;, = (n + 1)!/30, so formula (6.20) implies that 


(n+1)!) (n+)! 3 
n RK. Fo :~OS ST’ 1)! 
ae 3 30 Tie saul 


which proves our claim. | 


At this point, the reader should think about which entries of p correspond 
to vertices of T(p) that are of rank 1. 


6.3.2 Higher values of k 


If k > 2, then finding the total number of vertices of rank k is significantly 
more complicated. The main reason for this is that, in this case, then the 
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unique parent of a vertex of rank k— 1 does not have to be a vertex of rank k; 
it can be a vertex of rank ¢, where 0 < ¢< k. For instance, in the tree T'(p) 
shown in Figure 6.3, vertex 3 is of rank 1, and its parent, vertex 8, is also of 
rank 1. 


6.3.2.1 A simple, but useful Lemma 


Let a,,, be the total number of vertices of rank k in all decreasing binary 
trees of size n. It is then clear that an 441 < Qn,, since each vertex of rank 
k +1 must have at least one child of rank k. While finding the exact value of 
Gn,k is beyond the scope of this introductory discussion, the following lemma 
will turn out to be useful for us, even if its bound is far from being optimal. 


LEMMA 6.33 
For each positive integer k, there ezists a positive constant y, so that if n is 
large enough, then 


Qn,k 
| = Ve: 
n-n!} 


In other words, for any fized k, the probability that a randomly selected 
vertex of a randomly selected decreasing binary tree of size n is of rank k is 
larger than yz. In this early discussion, we will not try to find the best value 
of Yk- 

Before we prove Lemma 6.33, we need a simple notion. A perfect binary 
tree is a binary tree in which every non-leaf vertex has two children, and every 
leaf is at the same distance from the root. So a perfect binary tree in which 
the root is of rank has 1+2+---+2° = 2+! — 1 vertices. 

We will now compute the expected number of vertices p; that are of rank 
k, for which the subtree rooted at p; is a perfect binary tree. The expected 
number of such vertices is obviously a lower bound for the expected number 
of vertices of rank k. 

Let Q, be the probability that for a randomly selected permutation p of 
length 2*+1_— 1, the tree T(p) is a perfect binary tree (disregarding the labels). 
It is then clear that Qo = 1, and 

1 2 
Qre+1 = pert ek (6.21) 


So Qi = 1/3, and Q2 = 1/63. In particular, Q, is always a positive real 
number. 


PROPOSITION 6.34 

Let n > 2*+1—1, let p = pip2-+-pn be a permutation, and let us assume that 
Bri <i<n—2". (In other words, i is not among the smallest 2* indices 
or the largest 2" indices in p.) Let Py be the probability that the vertex p; of 
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T(p) is of rank k, and the subtree of T(p) rooted p; is a perfect binary tree. 


Then the equation 
2 


(Qh+1 + 1)2k+1 
holds fork > 1. In particular, P, is a positive real number that does not 
depend on n or 1. 


Pr = Qk: 


PROOF The subtree rooted at the vertex p; of T(p) will be a perfect 
binary tree with its root being a vertex of rank k if the following two inde- 
pendent events occur. 


1. The string pj; 4) of 2k+1 _ 1 consecutive entries of p whose middle entry 
is p; correspond to a decreasing tree that is a perfect binary tree, and 


2. All entries in pj;,,) are less than both entries bracketing pj,,), that is, 
both Pi—2k and Pi+tar- 


The first of these events occurs at probability Q;, and the second one occurs 
at probability OFAy ar? proving our claim. 
So Po = 1/3, and P; = 1/30, as we computed in the proofs of Proposition 
6.31 and Theorem 6.32. Furthermore, 
2 1 1 1 


Now we are in a position to prove Lemma 6.33. 


PROOF (of Lemma 6.33) Let V;(p) be the indicator random variable of 
the event that the subtree of p that is rooted at p; is a perfect binary tree 
whose root is of rank k. Then it follows from the definition of P, that 


If V(p) denotes the number of vertices of p that are of rank k and whose 
subtrees are perfect binary trees, then the linear property of expectation yields 


E(V(p)) = (n— 2°*")P,, 


since we do not allow i to be among the smallest 2 indices or the among the 
largest 2* indices. Therefore, (after division by n), the total number a,x of 
vertices at of rank k in all decreasing binary trees of size n satisfies 


k 
ai het gate Pe 
n-n! n 2 


for n > 2*+1, This completes the proof, since we can set 7, = Py /2. | 
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6.3.3 A System of Differential Equations 


In order to determine the exact value of an, for k > 2, we turn to exponential 
generating functions. We recall that the exponential generating function for 
the number of permutations of length n, and equivalently, decreasing binary 
trees on n vertices, is 1,59 M47 = 1/(1— 2). 

For k > 0, let Ax(z) = ae, Gn,k = denote the exponential generating 
function of the numbers of all vertices of rank k in all decreasing binary trees 
of size n. Let B;,(z) denote the exponential generating function for such trees 
in which the root is of rank k. In both A;(z), and By(z), we set the constant 
term to 0 since empty trees have no roots. Note that Aj,(z) = 0,5, #2", 
so in particular, the coefficient of z” in A, is the expected number of vertices 
of rank & in a randomly selected decreasing binary tree of size n. 


LEMMA 6.35 
The following differential equations hold. First, Bo(z) = z. Furthermore, 


Bile) = 2Bi-ale)- (<A 


1l-2z 
ifk>0. 


By(z) Bo(z) ater? Bi-a(2)) a By_1(z)? 


PROOF Let T be a decreasing binary tree counted by B,(z). Let us 
remove the root of T. On the one hand, this yields a structure counted by 
Bi,(z). On the other hand, this yields an ordered pair of decreasing binary 
trees such that one of them has a root of rank ks — 1, and the other one has 
a root of rank ¢, with € > k—1. By the Product Formula for exponential 
generating functions, such pairs are counted by the first product on the right- 
hand side. At the end of the right-hand side, we must subtract By_1(z)? as 
ordered pairs in which both trees have a root of rank k — 1 are double-counted 
by the preceding term. 


Example 6.36 
Let k = 1. Then Lemma 6.35 yields 


Bi(z) = 2Bo(z) - (; ! ) — Bo(z)? 


22 2 
~ l-gz : 


Therefore, using the equality B,(0) = 0, we deduce that 


1 3 
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LEMMA 6.37 
For k > 0, the linear differential equation 


Ay(2) = 2 - Ae(2) + Belz) 


holds. 


PROOF _ Let (T7,v) be an ordered pair so that T is a decreasing binary 
tree on n vertices, and v is a vertex of T that is of rank k. Now remove the 
root of T. If the root was v itself, then we get a structure counted by Bj,(z), 
just as we did in the proof of Lemma 6.35. Otherwise, we get an ordered pair 
(R,S) of structures, one of which is a decreasing binary tree, and the other 
one of which is an ordered pair of a binary search tree and a vertex of that 
tree that is at rank k. This explains the first summand of the right-hand side 
by the Product Formula. i 


Example 6.38 
Setting k = 1, we see that Ai(z) is the unique solution of the linear differential 
equation 


' 2 22 


with initial condition A,(0) = 0. This yields 


— 77-2 
z 


Ai(z) = —2 
U 


For larger values of k, we can theoretically proceed in the same way. We 
emphasize theoretically, because significant practical difficulties will arise as 
the solutions to our differential equations become more complicated. The 
following example shows some of these difficulties. 


Example 6.39 
Let k = 2. Then Lemma 6.35 yields 


n Zz. 23 
Bi(z) = nce) +4zIn(1/(1 — z)) - - mars 24 (6.22) 
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Now we can solve the differential equation provided by Lemma 6.37 with 
k = 2, to get 


Ao(z) = ame ty et aay 
= ee In(1/(1 — z)) + — a an In(1/(1 — z)) + a 
= = In(1/(1 — 2)) + — - = In(1/(1 — z))? 
+ Zin(1/(1 — 2)? — ee — Zinta — 2) 


While Ag(z) has many summands in the above form, it is not difficult to 
prove that the coefficients of at : ao grow faster than the coefficients of 


—z 
all other summands. Therefore, as n goes to infinity, the equality 
Gn 1721 


= —*) ~ 02124691 
eae age 


C2 = 


holds. 

This result raises many questions. In general, what kind of function will 
A3(z) be? Will it always be a sum of many summands, one of which will have 
significantly larger coefficients than the others? Will that summand always 
be a rational function? We have to introduce a new notion to address these 
questions. 


6.3.3.1 A class of functions 
Let PL(z) be the class of functions f : R — R which are of the form 


fe) => a(t - 2) In (+) (6.24) 


i=l 


where the coefficients a; are rational numbers, while the exponents 6; and c¢; 
are non-negative integers. Roughly speaking, PL(z) is the class of functions 


1 ” 
1l-z 


Note that the class PL(z) would not change if in its definition (6.24) we 
replaced (1 — z) by z. We use form (6.24) because that form is more directly 
applicable in the situations that will arise in this section. 

A few facts about PL(z) that are straightforward to prove using integration 
by parts will be useful in the upcoming discussion. 


that are “polynomials in 1 — z and In 


PROPOSITION 6.40 
The class PL(z) is closed under integration with respect to z. 
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PROOF We need to show that f(l—z)?In( 4 ) dz € PL(z). We prove 


1—-z 
this by induction on c, the initial case of c = 0 being obvious. Integration by 
parts yields 


fo-an (+) dz =—In (4) (6.25) 


+f? -eln (4) dz. (6.26) 


By the induction hypothesis, the integral on the right-hand side is in PL(z), 


proving our claim. 


PROPOSITION 6.41 
Let b>0 and c= 0 be integers. Then 


1 Cc 
fo — z)?In (=) dz = (1—z)"™ - g(z) + p(z), 
—2Z 
where p is a polynomial function, and g(z) € PL(z). The integral on the 
left-hand side is taken with constant term 0. 


PROOF Induction on c, the initial case being that of c = 0. If b = 0, 
then the statement is true, since [1 dz =a = (1—z)-(-1)+1. Ifb>0, 
then the statement is true, since f(1— z)> dz = (1—z)?t!. s4. 

The induction step directly follows from (6.25) and from the induction 


hypothesis. | 


LEMMA 6.42 
For all k > 0, we have 
B,(z) € PL(z). 


PROOF We prove the statement by induction on k. It is obvious that 
Bo(z) = z, and we saw in Example 6.36 that By(z) = 2In ( L ) gd 


1—z 3 
So the statement is true for k = 0 and k = 1. Now let us assume that 
the statement of the lemma holds for all positive integers less than k. It then 
follows from Lemma 6.35 that the summands of Bj,(z) are all in PL(z), except 


1 1 
1-z 1-z 


number and c; is a non-negative integer. The integral of each such summand 
is in PL(z) by Fact 10.2, and integrals of the other summands (those that are 
in PL(z)) are in PL(z) by Proposition 6.40. Therefore, as PL(z) is closed 


under addition, our claim is proved. 


possibly some summands of the form a;- 


-In ( ) , where a; is a rational 
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While the power series A,(z) are in general not in PL(z), the following 
weaker statement does hold for them. 


THEOREM 6.43 


For all nonnegative integers k, we have 


Px(2) 
Ke) = PEE + 2), (6.27) 
where f(z) € PL(z), and px(z) is a polynomial function with rational coeffi- 
cients that is not divisible by (1 — z). 


PROOF Lemma 6.37 provides a linear differential equation for A,(z). 
Solving that equation, we get 


3 Jf Bi — 2)? dz : C 


rt 


(6.28) 


where the integral on the right-hand side is meant with 0 as constant term. 
We saw in the proof of Lemma 6.42 that the summands of Bj(z) are all 


Ci 
in PL(z), except possibly some summands of the form a; + <4; - In (4) ; 


Therefore, the summands of (1 — z)?Bj(z) are all in PL(z). Even more 
= i 
with b; > 1. Therefore, Proposition 6.41 implies that the integral of each 
summand is of the form (1 — z)"**1g;(z) + pyy(z), where g;(z) € PL(z), and 
p(iy(z) is a polynomial function with rational coefficients. As b; + 1 > 2, this 
implies that f{ Bj,(z)(1 — 2)? dz = (1 — z)?9(z) + ax (z), where g(z) € PL(z) 
and g,(z) is a polynomial function with rational coefficients, and our claim is 


strongly, each summand of (1 — z)?Bi,(z) is of the form a;(1— 2)’ In ( 


proved. 


Determining the value of c3 is conceptually the same as determining cz. 
However, the computation becomes much more cumbersome. Lemma 6.35 
provides a formula for B5(z) as a sum. According to Maple, that sum has 
52 summands of the form a;z?! (In(1/(1 — z)))“. Using that expression for 
Bi(z), we can compute A3(z) using Lemma 6.37. Maple obtains a solution 
that has 59 summands. However, only 17 of these 59 summands contribute 
to p3(z), and therefore, only these 17 summands influence c3. The value we 
obtain for c3 is 


250488312501647783 


= —__—___—__ w 0.1091543117. 
2294809 143026400000 Deon ean nr 


C3 

Details of this computation can be found in [71]. 
While larger values of k could theoretically be handled in the same way, in 
the practice these larger values of k lead to differential equations that simply 
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contain functions with too many summands for Maple to handle. However, 
alternate forms of those equations can be obtained with some trickery. That 
allowed Boris Pittel and the present author to compute the values of cq, and 
cs in [76]. They obtained that c4 equals 


122058464141653662196290113232646304412999902283512425580156787323 
3353377025022449199852900725670960067418280803797231788288000000000’ 


a fraction whose denominator has 67 digits, and whose approximate value is 
0.0364. The approximate value of cs is 0.0074. Furthermore, in its simplest 
form, the rational number cs has a denominator that consists of 274 digits. 

Note that aan c; © 0.9975, meaning that for large n, about 99.75 percent 
of all vertices of all decreasing binary trees of size n are at distance five or less 
from the closest leaf. 


6.4 Variance and Standard Deviation 


While the expectation of a random variable X contains information about the 
average behavior of X, it does not describe how much the different values of 
X can differ. A constant variable can have the same expectation as one that 
does not take the same value twice. If we want to obtain information about 
the behavior of X from this more subtle viewpoint, we must use more subtle 
operators, such as variance. 


DEFINITION 6.44 _ Let X be a random variable. Then 


Var(X) = E((X — E(X))”) 


is called the variance of X. 


Example 6.45 
Let X(p) be number of 2-cycles of a randomly selected n-permutation p, and 
let n > 4. Then 


Var(X) = . 


SOLUTION _ The following observation that is immediate from the lin- 
earity of expectation is often useful in variance computations. 


Var(X) = E(X*) — 2E(X)E(X) + E(X)? = E(X?) -— E(X)*. (6.29) 
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In our case, Example 6.27 shows that E(X)? = 1/4. So all we have to do is 
to compute E(X?). 

In order to accomplish that, define the indicator random variables X;,; as 
in Example 6.27. Note that X?; = Xi,;, and therefore, E(X?,;) = E(Xi,;). 
Then we have 

2 


E(X?)=E eae cr = So BX) + So B(XigXvy) 
(4,9) C[n] (4,9)E[n] {iF ALU 5} 


=E(X)+ So E(XijXi), 
{4,9}{09 FCI] 
(dh G/ F=0 
where the second term is explained by the fact that no entry of p can be part 
of more than one cycle. 

Finally, we have to compute E(.X;,;X;j,;/) in the case when (i, j) and (2’, 7’) 
are disjoint sets. In this ae we have P(X;,j;Xi77 = 1) = (n—- 4)!/n); 
therefore, E(X4,;,Xv,j’) = coe (ordered) 
disjoint pairs of 2-element sects of [n] is (3) ("5°). Substituting these into 
the last equation, we get 


E(X?) = ; a Me = ) a a : 


Var(X) = E(X?) — E(x)? = = 


Therefore, 


Note that if 2 <n < 4, then Var(X) = as since in that case, p cannot 
have two disjoint 2-cycles, and the term Ue oT Ge 1s -) equals 0, not 1/4. 


Finally, for n = 1, we obviously have E(X) = Var(X) = 0. 


Example 6.46 

Let Y(p) be the number of descents of the n-permutation p = p1p2-+-- Pn, and 
let n > 2. Then 

n+1 


Var(Y) = 13 


SOLUTION Define the indicator variable Y; : 5, — R by 
1 if Pi > Pi+1 ; 


Yi(p) = 
0 if not. 
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It is then clear that 37", Y¥, = Y, that E(Y;) = 1/2, and that E(¥?) = 1/2. 
It is immediate from Example 6.24 that E(Y) = (n —1)/2. Therefore, we 


have 
(n—1)? 


Var(¥) = B(¥?) - BY)? = B(v?)- 4, 


and 


BY") =E (S¥) = So By?) + 2 2BUY;) 


i<j 
ere n—-2 
se 2 ED) on » 2E(Yi¥i+1). 
ye wz 


In the CS) pairs where i < j — 1, the probability that i and 7 are both 
descents is 1/4. In the pairs where 7 = i + 1, this probability is 1/6 as only 
one of the six possible patterns on the entries p;, pi+1, Pit+2 has the required 


property. Substituting this into the last equation, we get 


E(Y?) = a SAIN) elie 


Therefore 


Var(Y) = 


The variance of X is always nonnegative, as it is the expectation of a non- 
negative variable. This makes the following definition meaningful. 


DEFINITION 6.47 The standard deviation of the random variable Y 


is the value \/ Var(Y). 


If there is no danger of confusion as to which variable is referred to, the 
standard deviation is often denoted by oc. 

The following classic theorem shows that it is unlikely for a variable to differ 
from its expectation by a multiple of its standard deviation. 


THEOREM 6.48 
[Chebyshev’s Inequality] Let X > 0, and let X be any random variable that 
has a finite range. Then the inequality 


i 
P(X — u| 2 Ao] S 55 
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holds. 


PROOF Note that 


B((X = 4)?)= Sie PUK =n) == YD i PX-wP= 4 


i>r20? 


as the last term is obtained by omitting some non-negative summands from 
the previous one. Since each summand in the last term satisfies i > 27, this 
implies 


a >No? So Pl(X =p)? =i] =o? - PIX = pl > Ao]. 


i> 202 
Dividing by \?c7, we get the inequality that was to be proved. | 


Chebyshev’s inequality is useful when the standard deviation of a variable 
is small compared to its expectation, or compared to its maximal value. 


Example 6.49 

For every positive €, and every positive a, there exists an integer N so that 
ifn > N, then the number of n-permutations that have less than (0.5 — a)n 
descents or more than (0.5 + a)n descents is less than €- n!. 


SOLUTION _ Let p be arandomly selected n-permutation, and let X (p) = 
d(p). Then E(X) = (n—1)/2, and Var(X) = \/(n + 1)/12. So the variance is 
of a smaller order of magnitude than the expectation, and therefore, constant 
multiples of the variance are still very small compared to the expectation. 
Chebyshev’s inequality implies that for all real numbers A, we have 


P [|X = ul >A VF D2] Z > (6.31) 


Now let us select 4 so that xr <. Then select N so large that ifn > N, then 
an > A,/(n+1)/12. Then P [|X — p| > an] is less than the left-hand side of 
(6.31), while ¢ is larger than the right-hand side of (6.31). | 


The phenomenon described by Example 6.49 is often referred to by say- 
ing that the number of descents of a permutation is concentrated around its 
mean. In general, Chebyshev’s theorem implies that a variable is concentrated 
around its mean if its standard deviation is of a smaller order of magnitude 
than its mean. 
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6.4.1 Application: Asymptotically Normal Distributions 


While this book only discusses discrete probability, the reader must have seen 
the definition and basic notions of continuous probability in a Calculus class. 
We will therefore not repeat the basic definitions here. Recall that the variable 
Z has normal distribution if it has density function 


1 


oV 20 


en (a—u)?/ (207). 


f(z) = 


Here yp is the mean and o? is the variance of Z. In this case, the graphs of 
f(x), and its antiderivative F(a) (the distribution function of Z) look normal 
as can be seen in Figure 6.13. 


Normal Density Normal Distribution 


FIGURE 6.13 
Normal density and normal distribution. 


Why do we mention this in a book about discrete combinatorial objects? 
The answer is because a notion of convergence makes normality relevant for 
sequences of discrete distributions as follows. 

We say that the sequence of variables Z,, converges to the variable Z in 
distribution if 

Jim, F(a) = F(a) 
for every real number a where F is continuous. Here F;,(a) (resp. F'(a)) is the 
distribution function of Z, (resp. Z). That is, F,(a) = P[X, < a]. If u=0 
and og = 1, and Z has normal distribution, then we say that Z has standard 
normal distribution. 

Let X,, be a random variable, and let a,(k) be a triangular array of non- 
negative real numbers, n = 1,2,---, and 0 < k < m(n) so that 


An (k) 


P[|X, = k| = pp (k) = ——-———_.. 
Bin = Pol) = Ste) a C0) 
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Set gn(z) = ai) pn(k)z*. In other words, gn(z) is a generating function 
satisfying g,(1) = 1. Such generating functions are sometimes called proba- 
bility generating functions. 

Let us introduce the following notation that is standard in most probability 
texts. 


e Let Z=Z-—E(Z), 
e Let Z = Z/,/Var(Z), and 


e Let Z, > N(0,1) mean that Z,, converges in distribution to a variable 
with standard normal distribution. 


The following theorem, which is proved in most probability textbooks, con- 
nects the real roots-only property to normal distributions. 


THEOREM 6.50 
Let Z, and gn(z) be as above. If gn(z) has real roots only, and 


On = V Var(Zn) > &, 


then Z, —> N(0,1). 


Example 6.51 
Let Z,, be the number of cycles of a randomly selected permutation of length 
n. Then Z,, converges to a normal distribution, that is, Z,, + N(0,1). i] 


SOLUTION _ Recall from Chapter 3 that 
1 
gn(2) = 22+ 1)--(2+n-1), 


so the real roots criterion holds. Furthermore, you are asked to compute op, 
in Exercise 35. Once o, is computed, it becomes obvious that ao, > oo, 
completing the proof. 


The fact that the distribution of the number of cycles of a random n- 
permutation converges to a normal distribution is sometimes referred to by 
the sentence, “the number of cycles of a random permutation is asymptotically 
normal” . 

The reader is invited to use Theorem 6.50 to prove that the number of 
descents of a random permutation is asymptotically normal. This implies 
that the same holds for all Eulerian permutation statistics. A somewhat more 
advanced application of Theorem 6.50 shows that the number of blocks of a 
random partition of the set [n] is asymptotically normal. 
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Finally, we point out that the real zeros property is not necessary for asymp- 
totic normality to hold. An example for this is d-descents of permutations, 
which are defined in Exercise 14. Inversions of permutations are a special case 
of d-descents. When the real zeros property does not hold, but o, is large 
in some sense, a logical next step to prove asymptotic normality is often the 
Janson dependency criterion. That criterion, and some of its applications to 
permutation enumeration, can be found in [62] and [67]. The main idea is 
that if the probability generating function of Z,, has real roots only, then Z,, 
can be decomposed into the sum of some independent random variables. If 
such a decomposition is not possible, but Z,, can be decomposed into the sum 
of some random variables amongst which there are not too many dependency 
relations compared to Var(Z,,), then the Janson dependency criterion may be 
applicable. 


(I 


6.5 Application: Longest Increasing Subsequences 


Let p be a randomly selected n-permutation, and let X,,(p), or, when there 
is no danger of confusion, X(p) denote the length of the longest increasing 
subsequence of p. What can we say about E(X)? 

We start with a classic result due to Erdés and Szekeres that will provide 
us with a lower bound for E(X). 


PROPOSITION 6.52 
Letn =km+1. Then any n-permutation p contains either an increasing 
subsequence of length k +1, or a decreasing subsequence of length m+ 1. 


PROOF _ Extending the idea of Theorem 4.21, we define the 2-way rank 
of an entry of a permutation. The 2-way rank of the entry p; of p is (r,s) if 
the longest increasing subsequence of p ending at p; is of length r, and the 
longest decreasing subsequence of p ending at p; is of length s. For example, 
the 2-way rank of the sixth entry of 7215436 is (2,4). 

It is clear that no two entries of p can have the same 2-way rank as the 
rightmost of two entries would have higher first coordinate or higher second 
coordinate than the 2-way rank of the leftmost one. Therefore, all n = km+1 
entries of p must have different 2-way ranks. Since there are only km distinct 
2-way ranks with r € [k] and s € [ml], the proposition is proved by the Pigeon- 
hole Principle. 
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COROLLARY 6.53 
For all positive integers n, the inequality 


B(X)>5-va 


holds, where the expectation is taken over all n-permutations. 


PROOF For any n-permutation p, either p or its reverse p” has an 
increasing subsequence of length at least \/n by Proposition 6.52. Therefore, 


X(p) + X(p") > Jn. 


A little algebraic manipulation can lead to a better lower bound. We know 
that for any n-permutation p, 


max r(p;) - max s(pj) >; 
ven jen 


otherwise, we could not have n different 2-way ranks in p. For any positive 
real numbers a and 6, we have a + b > 2V/ab, and therefore 


maxr(p;) + max s(p;) > 2/n, 
jen 


ten 
proving (after taking reverses), the following stronger corollary. 


COROLLARY 6.54 
For all positive integers n, the inequality 


E(X) > Vn 
holds, where the expectation is taken over all n-permutations. 


The problem of determining E(X) more precisely has been the subject of 
vigorous research in the last eighty years. The above results should at least 
provide an intuitive justification as to why the following question, also known 
as Ulam’s problem, was asked in this form. 


QUESTION 6.55 Let n go to infinity. Does the limit 


lim E(Xn) 
n—0oo Jn 


(6.32) 


exist, and if yes, what is it? 


Corollary 6.54 shows that if (6.32) exists, then it is at least 1. The existence 
of (6.32) was first proved by Hammersley [210]. Numerical evidence then 
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seemed to suggest that this limit is probably close to 2. This result has 
originally been proved in two parts, by Logan and Schepp [245] showing the 
lower bound limy_+.. eae > 2, and by Vershik and Kerov [319] showing the 


E(Xn) < 9, 


n 
Knowing the expectation of X, only begins the story. The solution of 


Ulam’s problem opened the door to even deeper research of the distribution 
of X,. The interested reader should consult [11] for a survey of results, and 
[28] for some more recent, spectacular improvements in this area. 


upper bound lim,-5., 


(I 


Exercises 


1. Let i, j, k, and @ be four elements of [n], and let p be a randomly selected 
n-permutation, where n > 4. 


(a) What is the probability that the four elements are all in one cycle 
of p? 

(b) What is the probability that these four elements are in four different 
cycles? 


2. (-) What is the probability that the three largest entries of an n- 
permutation are in the same cycle? 


3. Continuing Exercise 1, what is the probability that the four elements 
are 


(a) in two different cycles? 


(b) in three different cycles? 


4. Find the expectation and variance of the number of excedances of ran- 
domly selected n-permutations. 


5. Let n > k. Find the expected number of k-cycles in a randomly selected 
n-permutation. 


6. Find the variance of the number of k-cycles of randomly selected n- 
permutations. 


7. Let Z(p) be the number of inversions of the n-permutation p, and let 
n > 2. Compute the variance of Z. 


8. Let U(p) be the number of weak excedances of the n-permutation p. 
Compute E(U) and Var(U). 


9. Is there a Ferrers shape on 20 boxes that has three hooks of length 7? 
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10 


11. 


12. 
13. 
14. 


15. 


16. 


17. 


18. 


19. 


20. 


21. 


Let X be the number of leaves of a randomly selected rooted plane tree 
on n+ 1 vertices. Find E(X). 


We want to build a tree with vertex set [n] satisfying as many constraints 
as possible from a finite set of constraints. The constraints are all of the 
type {(a, b)(c,d)}, meaning that the unique path in the tree connecting 
a to 6 should not intersect the unique path connecting c to d. 


Prove that no matter how many constraints we have, we can always find 
a tree having vertex set [n] that satisfies at least one third of them. 
What is the average number of alternating runs of all n-permutations? 


Prove Markov’s inequality. 


Let X be defined on the set A of all 1234-containing n-permutations, 
and let X(p) denote the number of 1234-copies in p. Let Y be defined 
on the set B of all 1324-containing n-permutations, and let Y(p) denote 
the number of 1324-copies of p. What is larger, E(X) or E(Y)? 


Let Y(p) denote the length of the cycle containing the entry 1 in a 
randomly selected involution of length n. Find E(Y). 


Find a noninductive proof for the fact that the average number of cycles 
of a randomly selected n-permutation is H(n) = S7j_, . 

Let a be a positive real number, and let p be a randomly selected n- 
permutation. Prove that 


Pliw)-3(5) >a(3)] +0 


as n goes to infinity. 


Let X,, be defined on the set of derangements of length n, and let X,,(p) 
be the number of cycles of p. Prove that for n > 4, we have 


(n—1)D(n—2)E(Xp—2 +1) + (n—-1)D(n—1)E(Xp_1) = D(n)E(Xn). 


Let Y,, be defined on the set of derangements of length n, and let Y;,(p) 
be the size of the cycle containing the maximal entry of p. Find a 
recurrence relation for E(Y,,). 


We know from Section 1.2 that for any fixed n, the sequence G(n, k) is 
unimodal. Where is the peak of that sequence, that is, for which k is 
G(n, &) maximal? 


Let p and q be two randomly selected n-permutations, and let Gp q be 
the graph defined in Problem Plus 5. Let Z(p,q) be the number of 
Hamiltonian cycles in G,,,. Prove that 
1 
F564 ons 


n 
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22. 


23. 


24. 


25. 


26. 


27. 
28. 
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Let X be a random variable whose range is finite, and assume that X 
takes only nonnegative values. Prove that 


= Var(X) 
PIX =01< aa 


Find an alternative proof for the result of Example 6.27 using the result 
of Example 6.3. 


We extend the notion of descents to Standard Young Tableaux as follows. 
We say that 2 is a descent of the SYT of P if i+ 1 appears in a row of 
P that is strictly below the row containing 7. So for instance, the SYT 
consisting of one single row has no descents, while the SYT on n boxes 
consisting of one single column has n — 1 descents. 


Let n be a fixed positive integer. For 7 € [n— 1], let A; be the event that 
z is a descent in a randomly selected SYT on n boxes. Find a short, 
direct proof of the fact that P[A,] is independent of 7. Do not use the 
result of the next exercise. 


Let A be a Ferrers shape on n boxes. Let 7 € [n — 1], and let A;,, be the 
event that 7 is a descent in a randomly selected SYT of shape A. Prove 
that P[A;,,] is independent of the choice of i. 


A deck of cards is bijectively labeled by the elements of [n], and is 
originally arranged in increasing order of the labels. We split the deck 
into a smaller decks of consecutive cards, with empty decks allowed, 
then we randomly merge the small decks together. Note that before 
merging, the smaller decks contained their cards in increasing order. 
This sequence of operations is called a riffle shuffle. Prove that the 
probability that we obtain a given permutation p € Sp, is 


(At re) 
Pan{p| = 2, 
where ri(p) denotes the number of rising sequences of p. A rising se- 
quence is an increasing subsequence of consecutive integers. For in- 
stance, 1324756 has three rising sequences, namely, 12, 3456, and 7. 


Deduce Theorem 1.8 from the result of the previous exercise. 
(+) A set F' of n-permutations is called min-wise independent if for all 
X C [n], and all  € X we have 


1 
Pimin{p(X)} = pe] = 5, 
|x| 
where p = p1p2---Pn is arandomly selected permutation in F’. That is, 
each element of X is equally likely to be the index of the smallest entry 
among the entries indexed by X. 
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29. 


30. 


bl. 


32. 


33. 


34. 


35. 


Small families of min-wise permutations are important tools in efficiently 
detecting identical or near-identical websites. Prove that there exists a 
min-wise independent family of size less than 4”. 


Let F be a min-wise independent family of n-permutations. Find a 
lower bound for |F'. 


Let F be a min-wise independent family of n-permutations, and let 
i € [n] be a fixed integer. Let X C [n], and let x € X. Denote P, the 
probability that for a randomly chosen permutation p = pp2---Pn € F, 
the entry pz is the ith smallest among the entries in the set p(X). Prove 
that 


Note that the definition of min-wise independence automatically assures 
this in the special case when i = 1. 


Let X(p) denote the number of very tight copies of the pattern 12 in a 
randomly selected n-permutation. (In other words, X(p) is the number 
of indices i so that in p = pip2--:pn, we have pi41 = p; +1.) Find 
E(X). 


Let X(p) be the smallest index 7 that is a descent of the n-permutation 
p, and set X(p) = nifp=12---n. Find E(X) and Var(X) if n is fixed. 


(—) Stirling permutations were defined in Exercise 42 of Chapter 1. Let 
Pp = Pip2:++ Pan be a Stirling permutation of size 2n. Let us say that 
i is a plateau of p if p; = pi41. Let us say that 7 is an ascent of p if 
i= 0 or if pj < pj41. Finally, let us say that 7 is a descent of p if 1 = 2n 
or pi > pizi. (We can imagine a 0 at the front and end of p, in order 
to motivate counting i = 0 as an ascent and i = 2n as a descent.) Let 
P£(p) (resp. A(p), D(p)) denote the number of plateaux (resp. ascents, 
descents) of p. Prove that 


2D =F =a 


Let Xn(p) denote the number of odd cycles of the randomly selected 
n-permutation p, and let Y;,(p) denote the number of randomly selected 
even cycles of p. Compute 

lim E(Xn—Yn)- 


n> Co 


Let X (p) be the number of cycles of the randomly selected n-permutation 
p. Find Var(X). 
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37. 


38. 


39. 


40. 


41. 


42. 


43. 
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. (+) Let X(p) be the number of peaks of the permutation p. Let E(X) 


be the expectation of X taken over all 132-avoiding n-permutations. 
Compute F(X) and compare the result with the corresponding result 
for all n-permutations. 


Consider all c,_1 rooted plane unlabeled trees on n vertices, and choose 
one of their total nc,_1 vertices uniformly at random. 


(a) What is the probability a, that the selected vertex is a leaf? 


(b) Interpret the result of part (a) in terms of permutations. 


Continuing the previous exercise, what is the probability that the se- 
lected vertex is a neighbor of a leaf? 


Consider all rooted plane trees whose vertices are bijectively labeled 
with the elements of [n] so that each vertex has a label that is smaller 
than the label of its parent. Let T;, be the number of such trees. Find 
an explicit formula for T),. 


Let ¢,, be the total number of leaves in all T;, trees of size n defined in 
Exercise 39. Find an explicit formula for @,. 


(+) Decreasing nonplane 1-2 trees were defined in Exercise 15 of Chap- 
ter 1, where we proved that they are counted by the Euler numbers. 
Consider all vertices of all E,, such trees on vertex set [n], and choose 
one vertex uniformly at random. Let a, be the probability that the 
selected vertex is a leaf. Find limy_..6 dn. 


(+) Continuing the preceding exercise, let b,, the probability that the 
chosen vertex is a neighbor of a leaf. Find limp, bn. 


Let X(p) be the number of all inversions in the permutation p. Compute 
E(X) over the set of all 312-avoiding permutations of length n. 


Let X(p) be the number of all inversions in the permutation p. Compute 
E(X) over the set of all 321-avoiding permutations of length n. 


(SE 


Problems Plus 


1, 


Let 7 € [n— 2], and let A be a given Ferrers shape on n boxes. Let X; be 
the indicator variable of the event that both i and +1 are descents in a 
randomly selected SYT of shape A. Is it true that E(X;) is independent 
of i? (Descents of SYT are defined in Exercise 24.) 
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10. 


11. 


Let X; be defined as in the previous Problem Plus. Find a formula for 
E(X;). 


Let 1 <j <i-—1< n—2, and let X;; be the indicator variable of 
the event that both 7 and 7 are descents in a randomly selected SYT of 
shape X. Is it true that E(X;,;) is independent of 7? 


Let X;,; be defined as in the previous Problem Plus. Find a formula for 
E(Xi,;)- 


Let p = pip2:::pn be an n-permutation, and let Gp be the directed 
graph on vertex set [n] in which there is an edge from 7 to j if p; = J. 


Now let p and q be two randomly selected n-permutations, and consider 
the graph Gq on vertex set [n] whose edges are the edges of G, and 
the edges of G,. Let n go to infinity. What is the probability that Gp, 
contains a directed Hamiltonian cycle? (A Hamiltonian cycle of a graph 
is a cycle that contains all vertices of the graph.) 


Keep the notation of the previous Problem Plus, and let p, g, and r 
be three randomly selected n-permutations. Let Gp.q be the graph 
that consists of the edges of G,, G,, and G, on vertex set [n]. Let n 
go to infinity. What is the probability that Gp, contains a directed 
Hamiltonian cycle? 


What is the expected number Eg,, of descents in a randomly selected 
SYT of a given shape A on n boxes? 


We have defined descents in Standard Young Tableaux in Exercise 24. 
Continuing that line of thinking, we can extend the notion of major 
indices to SYT by setting maj(T) = 0 iepcr) t- Here D(T) denotes the 
set of all descents of the Standard Young Tableaux T’. 


What is the expectation Emaj,, of the major index in a randomly se- 
lected SYT of a given shape A on n boxes? 


Let p be a permutation obtained by the riffle shuffle algorithm given in 
Exercise 26. Find a formula for the probability that p is of a given type. 


Let a = 2, and let us repeat the riffle shuffle algorithm of Exercise 26 m 
times. This means that starting from the second application, the smaller 
decks will not necessarily contain their cards in increasing order. Let p 
be a fixed n-permutation. Find a formula for the probability that this 
procedure results in p. 


Stirling permutations were defined in Exercise 42 of Chapter 1. Let 
D,(p) denote the number of descents of the Stirling permutation p of 
size 2n. Find an explicit formula for Var(D,,). 
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13. 


14. 


15. 


16. 


VT. 


18. 
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Let A,(z) = oe zP(), where p ranges all Stirling permutations of size 
2n. 


(a) Prove that A,(z) has real roots only. 


(b) If we select a Stirling permutation p of size 2n at random, what is 
the most likely value for D,,(p)? 


Let us select a random element (p,q) of the direct product S, x Si. 
What is the probability that p and q have the same number of cycles? 


Let p = pip2--:pn be a permutation. We say that the pair (i,7) is a 
d-descent in p if i <j <%7+d, and p; > p;. In particular, 1-descents 
correspond to descents in the traditional sense, and (n — 1)-descents 
correspond to inversions. For fixed n and d, let X,,4(p) denote the 
number of d-descents of the n-permutation p. Find Var(p). 


Let qg and q’ be two permutation patterns of length k, and let X,,,4(p) 
(resp. Xn,q/(p)) denote the number of copies of g (resp. gq’) contained 
in p. 


(a) Prove that the equality Var(Xn,q) = Var(Xn,q’) does not necessar- 
ily hold. 

(b) Prove that for all patterns q of length k, there exists a constant cz, 
so that Var(Xn,q) = cpn?*-1 + O(n?*-?). 


Let q be a 132-avoiding permutation pattern of a fixed length k, let X,(p) 
be the number of copies of g in the n-permutation p, and let E(X,) be 
the expectation of X, taken over all 132-avoiding permutations of length 
n. 


For which q will E(X,) be minimal? For which q will E(X,) be maxi- 
mal? 


Keep the notation of the previous problem. Prove that 
E(X312) = E(X231) = E(Xa213). 
Try to find a bijective proof. 


Let p be a 132-avoiding permutation, and let T(p) the decreasing binary 
tree of p. Note that as p is 132-avoiding, we can remove the labels of 
T(p) without losing any information about p. Let us remove all edges 
from connecting left children to their parents from T(p). We call the 
remaining components, which are all paths descending to the right, the 
spines of p. The spine structure of p is the size of its spines, in descending 
order. See Figure 6.14 for an illustration. 


Prove that if p and q are two 132-avoiding permutations, and their spine 
structures are identical, then E(X,) = E(X,), where the expectations 
are taken over all 132-avoiding permutations of length n. 
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FIGURE 6.14 
The decreasing binary tree T(p) for p = 65783412 and its spines, leading to 
the spine structure (3,2, 1,1). 


19. Can the statement of the preceding Problem Plus be reversed? That is, 
if E(X,) = E(X,) for two 132-avoiding permutations p and gq, does it 
follow that p and q have identical spine structures? 


20. A k-alternating subsequence in a permutation p = p,p2--- Pn is a subse- 
quence pj, Pi. +++ Pi, So that 
Pix > Piz < Dis > Dig ees ) 


and, for all j, 
Pi, _ Pis41| >k. 


Let as,(p) be the length of the longest k-alternating subsequence of p. 
Find E(as;,), where the expectation is taken over all permutations of 
length n. 


a 


Solutions to Problems Plus 


1. Yes, this is true, and can be proved similarly to Exercise 25. See [214] 
for an analysis of all cases. 


2. Let X = (a1, a2,--- ,@a), and let the conjugate of A be denoted by 4’ = 


(a,a5,--- ,a@/,,). It is proved in [214] that 
a.(a’.—1)(a, —2 
E(X;) =dy = ale = 1G 2). (6.33) 


ae n(n — 1)(n — 2) 
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. Yes, this is true, and can be proved again similarly to Exercise 25. See 


[214] for details. 


. Let us keep the notation of the solution of Problem Plus 2. Furthermore, 


let us introduce the new notation 


ai (a; 1) (ax 2)(ay 3) 
n(n —1)(n— 2)(n — 3) ° 


Ex, = 
ADjrk>l 


It is then proved in [214] that 
E(Xij) =e — dy —dy +e, + ey, 
where cy, is defined in (6.34) below. 


. This probability converges to zero as n goes to infinity as is proved in 


[125]. 


. This probability converges to 1 as n goes to infinity as is shown in [180]. 


. Let the given shape 4 have rows of length aj, a2,--- , a, and columns of 


length a{,a,--- ,a),. It then follows from a more general result of Peter 
Hasto [214] that 


k 
n-1 ~ a; (a; — 1) ~ ai.(ai — 1) 
+= Ean = 1 —— -— + +— | (6.34 
Cr d,» 9 ats ra nn) 2 n(n — 1) ( ) 
ai(a’,— 1) 
iT (6.35) 
( ) re n(n — 1) 


Note that by the result of Exercise 25 and the linearity of expectation, 
we get that for a given i € [n — 1], 


n(n — 1) 


Pie DIT) =>~ he) 


J<t 


(6.36) 


where T is a randomly selected SYT of shape X. 


. It is straightforward from (6.36) and the linearity of expectation that 


where the a; are defined as in the solution of the previous Problem Plus. 
Note that here, just as in (6.34), the two sums within the brackets cancel 
if our Ferrers shape is self-conjugate. 
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9. 


10. 


11. 


12. 


It is proved in [147] that the probability that p is of type (m1, N2,--+ , Mn) 
is 
jatnj—l 
TTj=1 (# i ) 
a” , 


where fq is the number of aperiodic circular words of length j over 
an alphabet of a letters. In other words, fq is the number of ways to 
design aperiodic necklaces using 7 beads having colors 1,2,:--,a. The 
numbers fj, are fairly well-studied, and it is known that 


1 ' 
fja = ae p(d)ai/?, 


dj 


where d ranges all positive divisors of 7, and yz is the number-theoretical 
Mobius function, that is, (d) = 0 if d is divisible by a perfect square 
larger than 1, otherwise y(d) = (—1)*, where k is the number of distinct 
prime divisors of d. See [302] for this fact, and the definition of the 
Mobius function in a more general setting. 


It is proved in [31] that this probability is 


Pa m= oe) 


Qmn 
See Exercise 26 for the definition of ri(p). 


It is proved in [63] that Var(D,) = a, The main tool of the proof 


is the following recurrence relation. Let s, = E(D?). Then sys. = 
2n—1. Sy + An+4 
2n+1 ae 30° 


(a) This result is proved in [63] by an induction argument. We have 
Ai(z) = z, and Ao(z) = 227+ z = 2(2z +1). It is not difficult to 
prove that the recurrence relation 


An(z) = (2 — 2*)Al_(z) + (Qn — 1)zAn-1(z) (6.37) 
or, equivalently, 


Ay(z) = 2(1 - zn ((1 — z)*7?” An_1(z)) (6.38) 
holds. Let us assume that the roots of A,_1(z) are real, distinct 
and non-positive. Clearly, A,(z)) vanishes at z = 0. Furthermore, 
by Rolle’s theorem, (6.38) shows that A,,(z) has a root between 
any pair of consecutive roots of A,_1(z), providing n—1 real roots 
for A,(z). So the last root must also be real, since complex roots 
of polynomials with real coefficients must come in conjugate pairs. 
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13. 


14. 


15. 


16. 
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We must still show that the last root of Ap,(z) must be on the right 
of the rightmost root of A,-;. Consider (6.37) at the rightmost 
root 29 of An_1. As zo is negative, so is 29 — 23, and so A,(zo) 
and A!,_,(zo) have opposite signs. The claim now follows, since 
in —oo, the polynomials A,,(z) and A’,_,(z) must converge to the 
same (infinite) limit, as their degrees are of the same parity. As 
A! _,(%) has no more roots on the right of zo, the polynomial A, (z) 
must have one. 


(b) As A,(z) = ¥2, a;2* has real roots only, it follows from Theorem 
3.25 that if its largest coefficient is a;, then |i — E(D,,)| < 1. We 
computed in Exercise 33 that E(D,) = (2n+1)/3. So if (2n+1)/3 
is an integer, then the most likely value of D,,(p) is m = (2n+1)/3, 
otherwise it is one of the two integers bracketing m. 


This question was answered by Herb Wilf [831] who proved that the 
requested probability is asymptotically equal to wee The proof 


uses the fact that if g(z) = 772, ax2*, then 
n-1 1 Or : 
jal? = =f [g(a 
d, on J, (ete 


Wilf then applies the above formula to the generating polynomial 
pe Cn, k)z* = 2(24+1)---(z+n-—1) of the signless Stirling numbers 
of the first kind, and computes the integral on the right-hand side using 
an advanced technique called the method of Laplace for integrals. 


It is proved in [62] that if n > 2d, then 


_ 6dn + 10d? — 3d? — d 
-_ 72 ; 


while if n/2 <d<n-—1, then Var(X,,) equals 


Var(X;) (6.39) 


2n3 — 6n? + 4n — 12d3 — 21d? — 9d — 12nd + 24nd? + 30nd + 18 
72 : 

(6.40) 

(a) It is easy to verify that Var(X4,123) 4 Var(X4,132). 

(b) This result is proved in [67]. 


The minimum is taken for the increasing permutation and the maximum 
is taken by the decreasing permutation of length k. This family of 
questions was initiated by Joshua Cooper, and the proofs of these results 
(due to the present author) can be found in [66]. 
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17. 


18. 
19. 


20. 


The first equality is trivial (take inverses), but the second one is not. 
See [69] for both a straightforward computational proof and a more 
elucidating bijective proof. We mention that Cheyne Homberger [216] 
has proved the following. The average number of 231-copies in all 182- 
avoiding n-permutations is equal to the average number of 231-copies in 
all 12avoiding n-permutations. 


This result is due to Kate Rudolph [277]. 


Yes. This fact was conjectured by Kate Rudolph [277], and proved soon 
afterwards by Lynn Chua and Krishanu Roy Sankar [114]. 


Tommy W. Cai [107] has proved that E(as,) = Mae bts proving a 
conjecture of Drew Armstrong. 


Taylor & Francis 
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Permutations and the Rest. Algebraic 
Combinatorics of Permutations. 


7.1 Robinson—Schensted—Knuth Correspondence 


This chapter is devoted to the connections between permutations and various 
other objects in combinatorics. This is certainly a huge topic, and we can 
therefore only skim the surface of a few selected areas. Our goal is to give the 
reader an overview of some main lines of research to aid the decision of what 
literature to consult next. 

In the first section, we present the famous Robinson—Schensted—Knuth cor- 
respondence that connects the combinatorics of permutations and the com- 
binatorics of Standard Young Tableaux. There are several excellent books 
[185, 279], devoted entirely or mostly to the fascinating subject of Young 
Tableaux, and we will not try to parallel them. Instead, we will be concen- 
trating on those parts of the area that are most directly connected to the 
enumeration of permutations. 

The Robinson—Schensted—Knuth correspondence provides a direct bijective 
proof for Theorem 6.11, showing once again that the number of pairs of SYT 
of the same shape, consisting of n boxes each, is n!. This is achieved by 
a bijection risk from the set of all n-permutations onto that of such pairs. 
(There is no risk involved, but when pronounced, that word is shorter than, 
say, RSK.) In addition, the bijection has a very rich collection of interesting 
properties, such as turning natural parameters of permutations into natural 
parameters of SYT. 

To start, let 7 = 71 72--- 7» be an n-permutation. We are going to construct 
a pair of Standard Young Tableaux risk(7) = (P,Q). The two tableaux will 
have the same shape. The two tableaux will be constructed together, in n 
steps, but according to different rules. In the P-tableau, some entries will 
move after they are placed, while this will not happen in the Q-tableau. We 
will call our tableaux P and @ throughout the procedure, but if we want to 
emphasize that they are not completely built yet, we call them P; and Q; to 
show that only i steps of their construction have been carried out. 

We are going to describe the bijection risk step-by-step, and we will illus- 
trate each step by the example of 7 = 52314. 
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P Q 
5 2314 1 
FIGURE 7.1 
The situation after Step 1 of creating risk(52314). 
P Q 
314 : 
B) 2 
FIGURE 7.2 


The situation after Step 2 of creating risk(52314). 


Step 1 


Step 2 


Step 2 


Take the first entry of 7, and put it in the top left corner of the tableau 
P that we are in the process of creating. Then put the entry 1 in the 
top left corner of the tableau Q that we are creating, encoding the fact 
that the top left corner was the first position of P that was filled. See 
Figure 7.1 for an illustration. 


Now take 72, the second entry of 7. If m2 > 71, then simply put 72 to 
the second position of the first line of P. We then write 2 in the second 
box of the first line of Q, to encode that this was the box of P that got 
filled second. 


If m2 < 7m (as in our running example), then we cannot do this as the 
first row of P cannot contain entries in decreasing order. Therefore, in 
this case, 72 will take the place of 7, and 7, will descend one line, to 
take the first position of the second row of P. To encode this, we write 
2 to the first box of the second line of Q. 


See Figure 7.2 for an illustration. 


We then continue the process the same way. Let us assume that the 
first 7 entries of 7 have already been placed, and that we have created a 
pair (P;,Q;) of partial Standard Young Tableaux on 7 boxes each, and 
of the same shape. Now we have to place 7;,,. Generalizing the rules 
that we have seen in the special case of i = 2, we look for the leftmost 
entry y in the first row that is larger than 7;41. If there is no such entry, 
we simply put 7,41 at the end of the first line. If there is such an entry 
y, then 7:41 will take the position of the entry y, while y will descend 
one line, and will be inserted in the second row of P; according to the 
same rules. In other words, 7;+41 displaces the smallest entry y of the 
first row that is larger than 7;41. Note that this preserves the Standard 
Young Tableau structure, since the first row will remain increasing, and 
the column into which 7;+1 is inserted will remain increasing. Indeed, 
41 displaced an entry y that was larger than 7;41, and 7;+1 is larger 
than its left neighbor in the first row. 
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2|3 1 |3 1/3 3 
14 
5 2 4 |2 
5 4 
3\4 1 | 3/5 
2 
4 
FIGURE 7.3 


The situation after Steps 3, 4, and 5 of creating risk(52314). 


When y is looking for its place in the second row, the same rules apply 
to y as applied for 7;,1 in the first row. That is, if there is no element 
larger than y in the second row, then y will be placed at the end of the 
second row; otherwise, y will displace the smallest entry z in the second 
row that is larger than y, and this entry z will descend to the third row, 
to look for a position, subject to the same rules. For the same reasons 
as we explained in the previous paragraph, these steps will preserve the 
Standard Young Tableau structure of P;,,;. When this procedure ends, 
the resulting tableau P;;1 will have a box in a position where P; did 
not. We will write i+ 1 into that position in the Q-tableau, to encode 
the fact that that position was the (¢ + 1)st position of P to get filled. 
This preserves the Standard Young Tableau structure of the Q-tableaux, 
since 7+ 1 gets inserted at the end of a row and at the end of a column. 


Repeating this placement procedure n times, we get a pair (P,Q) = (Pr, Qn) 
of Standard Young Tableaux of the same shape, consisting of n boxes each. 
The fact that P,, and Q,, are indeed Standard Young Tableaux is easy to prove 
by induction since we have seen the insertion of each new entry preserves the 
Standard Young Tableau structure of both P; and Q;. The shapes of P and 
Q are identical, as in each step we added a new box to the same position in 
each of them. We then set risk(7) = (P,Q). 

See Figure 7.3 to see how the complete image risk(52314) is obtained. 


THEOREM 7.1 
The map risk defined above is a bijection from S,, to the set of pairs of Standard 
Young Tableaux (P,Q) having identical shape and consisting of n boxes each. 


PROOF _ It suffices to show that risk has an inverse, that is, for any pair 
(P,Q) of Standard Young Tableaux having identical shapes and consisting of 
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n boxes each, there exists exactly one 7 € S,, so that risk(7) = (P,Q). 

We prove our statement by induction on n, the initial case of n = 1 being 
trivial. 

In order to prove the inductive step, we show how to recover the last entry 
T of 7 from (P,Q). The position of the entry n in the Q-tableau reveals 
which position a of the P-tableau was filled last. As n is at the end of a row 
in Q, this position a is at the end of a row in P. If this is the first row, then 
there was no displacement involved in the last step of the creation of P and 
Q, and 7, is simply the content c(a) of position a. 

If a is at the end of row 7, then c(a) got to a after being displaced from its 
position 6 in row i—1. Fortunately, we can easily recover b. Indeed, if u was 
the entry that displaced c(a) from position b, then in the (i — 1)st row, c(a) 
was the smallest entry larger than u. Therefore, u is the largest entry in the 
(i — 1)st row that is smaller than c(a), and the position of u is b. Now we can 
argue as in the previous paragraph. That is, if 7 = 2, that is, b is in the first 
row of P, then u could not have come from a higher row, so u must have come 
directly from 7, forcing 7, = u. Otherwise, u got to b after being displaced 
from the (i — 2)nd row. In the latter case, we repeat the above argument to 
find the position in the (i — 2)nd row from which u was displaced, and the 
entry that displaced it, and so on. 

This procedure ends when we reach the first row and find out which entry 
started Step n of the tableau-creating procedure. That entry is, obviously, 
TW 

Once we have determined 7,, we remove the box containing 7, from P 
and the box containing n from Q. By the definition of the Q-tableaux, this 
leaves us with two Standard Young Tableaux P’ and Q’ on n — 1 boxes that 
are of identical shape. (The entries of P’ are not necessarily the elements 
of [n — 1], but that does not matter, as they are precisely the entries of the 
partial permutation 2’ = 7172---7, 1.) By our induction hypothesis, we can 
recover 7’ from the pair (P’, Q’), completing our induction proof. 


Example 7.2 

Let P and Q be as shown in Figure 7.4. Then the position of 8 in Q tells us 
that the last box to be added to P was the box containing 7. This is the box 
a of the above argument. Its content c(a) = 7 got there after being displaced 
from the end of row 2. In that row, it had to be at the end, so it was displaced 
by the entry 4. The entry 4 in turn had to be displaced from row 1, by the 
largest entry there smaller than 4. That entry is 3, so we have 7g = 3. 0 


If risk(7) = (P,Q), we will often write P(7) and Q(z) for the two tableaux 
of risk(z). 

As we have mentioned, many parameters of 7 are encoded in risk(7). The 
length of the longest increasing subsequence in 7 is, for example, very easy to 
read off risk(z). 
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FIGURE 7.4 


Recovering (P’, Q’) from (P,Q). 


LEMMA 7.3 
If the length of the longest increasing subsequence of 7 is k, then P(r) has k 
columns. 


PROOF _ Recall from the proof of Theorem 4.21 that 7; is called an entry 
of rank j if 7; is at the end of a subsequence of 7 that is of length 7, but there 
is no increasing subsequence of length j + 1 in 7 that ends in 7;. 

We prove a stronger statement by showing that the first position each entry 
of rank 7 of 7 takes during the construction of P(m) by the risk algorithm is 
the jth position of the first row. This will show that the length of the first 
row, therefore, the number of columns, of P(7) is indeed the maximal rank 
in 7. 

Our proof of this stronger statement is by induction on 7. The case of 7 = 1, 
corresponds to left-to-right minima. These entries must enter the first row at 
the first box as they are smaller than any entry previously placed. So the 
initial step is complete. 

Now let us assume that we know the statement is true for entries of rank 
j, and let us prove it for entries of rank 7 +1. Let x be such an entry. Let 2’ 
be the entry of rank 7 on the left of x that is closest to x. By our induction 
hypothesis, x’ entered the first row at box j7. So when x starts looking for 
its position in the first row, box j is either taken by 2’ or (at this point we 
cannot yet exclude it) another, smaller entry that displaced x’. In either case, 
box j contains an entry that is smaller than x, so x cannot enter the first row 
anywhere before position 7 + 1. 
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On the other hand, x cannot enter the first row anywhere after position 7+1 
either. Indeed, assume this happens; this implies that when « starts looking 
for a position, there is an entry y < x in position 7 + 1 of the first row. By 
our induction hypothesis, that entry cannot have rank less than 7 + 1. That 
is a contradiction, however, for we could affix x to the end of any increasing 
subsequence ending in y, yielding that the rank of x is at least 7 + 2. 


The result of Lemma 7.3 provides an obvious alternative proof of the fact 
that Av,(123---k) < (k—1)?". There is, however, a much more refined 
application of this result to permutation enumeration, one that we promised 
in Chapter 5. 


THEOREM 7.4 
For any fixed positive integer k, the sequence Av,(12---k) is P-recursive. 


Note that this is the only result known that proves Conjecture 5.4 for an 
infinite number of patterns. 


PROOF ~ We have just seen in Lemma 7.3 that permutations not having 
increasing subsequences of length k can be associated with pairs of Standard 
Young Tableaux having at most k—1 columns. In other words, 5;,(12--+k) = 
>> f#, where F runs through all Ferrers shapes of size n that have at most 
k — 1 columns, and fr denotes the number of Standard Young Tableaux 
of shape F. Let F = (mj,mg,--+ ,mg-1) be such a Ferrers shape. Then 
m, > mz >-+: > mg_1 > 0, and pry m;, =n, with m; denoting the size 
of the ith column. The hooklength formula now implies (see Exercise 23 for 
details) that 


(m4 + mg tere Mp1)! 


(Cs ee ey ee 


fr= I] @u-m+i-4 


1<i<j<k-1 


We can easily see that the right-hand side is P-recursive in each of its 
variables; therefore, by repeated applications of Lemma 5.10, it is also P- 
recursive in the sum of these variables. This implies that 


S- f2 = Avn(12--+k) (7.2) 
Mite +Mp,_1S=N 
is P-recursive in n. | 
This proof used the connection, established in Lemma 7.3, between the 


length of the first row of P(z) and the length of the longest increasing subse- 
quence of 7. There is a far-reaching generalization of this observation, proved 
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FIGURE 7.5 
The tableau P(261735984). 


by Curtis Greene and Daniel Kleitman, and, independently, by Sergey Fomin, 
who was 19 years old at that time. 


THEOREM 7.5 

[199] Let x be a permutation, let P(r) have k rows, and let a; denote the 
length of the ith row of P(m). Then for alli € [k], the maximum size of the 
union of i increasing subsequences in 7 is equal to ay +ag4+---+4a;. 


Example 7.6 
Let 7 = 261735984. Then P(7) is shown in Figure 7.5. 

We see from P(m) that (a1,a2,a3) = (4,4,1). On the other hand, the 
longest increasing sequence of 7 is of length four (2678), the largest union of 
two increasing sequences is of size eight (2678,1359), and the largest union of 
three increasing sequences is of size nine. 


Note that, in particular, Theorem 7.5 implies that if 7 and 7’ are two n- 
permutations so that P(z) = P(x’), then the maximum size of the union of 
i increasing subsequences in 7 is equal to the same parameter in p’. In other 
words, this parameter depends only on the P-tableau of a permutation. 


PROOF (of Theorem 7.5) This is a special case of the famous Greene— 
Fomin—Kleitman theorem for partially ordered sets. See [199] for a proof. A 


more accessible reference is [279]. | 


It seems that the increasing subsequences of 7 are quite well encoded by 
P(r). It is natural to ask whether there are similarly strong results about the 
decreasing subsequences of 7. Fortunately, the answer is in the affirmative, 
because of the following theorem of Schensted [280] describing the P-tableau 
of the reverse x” of x in terms of the transpose (the reflected image through 
the NW-SE diagonal) P()? of P(r). 


THEOREM 7.7 
For any n-permutation 7, the equality P(r)’ = P(x") holds. 
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FIGURE 7.6 

The P-tableaux of 31524 and 42513. 
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FIGURE 7.7 
Row and column insertion of 2 into P. 


Example 7.8 
Let 7 = 31524, then 7” = 42513, and the corresponding P-tableaux are indeed 
conjugates of each other as shown in Figure 7.6. 


We need to introduce some machinery developed in [280] before we can 
prove Theorem 7.7. Let P be a partial Standard Young Tableau, that is, a 
tableau that we have at some point as our P-tableau during the construction 
of risk(7). That is, P has rows and columns that are strictly increasing, but 
the integers written in the boxes of P can form any subset of [n], not just an 
initial segment. 

Now let us say the next entry of 7 to be inserted to P by the risk algorithm 
is x. The partial tableau that we obtain from P once z is inserted will be 
denoted by r,(P), where r refers to row insertion. Before you ask what other 
insertion we could possibly talk about, we define column insertion just as we 
did (row) insertion, except that instead of rows, we use columns. That is, 
the entry x to be inserted arrives into the first column, displaces the smallest 
entry a that is larger than 2x; this entry a then enters the second column, 
and proceeds analogously. The partial tableau obtained from P by column 
inserting x will be denoted by c,(P). 


Example 7.9 

Let a = 3512746, and let P be the partial tableau obtained after three steps 
of the risk algorithm. Then 2 = 2, and the tableaux P, r2(P) and co(P) are 
shown in Figure 7.7. 


The crucial property of row and column insertion is that they commute in 
the following strong sense. 
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PROPOSITION 7.10 
Let P be a partial tableaux, and let u and v be two distinct positive integers 
that are not contained in P. Then we have 


Gul ty(P)) = fol GsP)): 


The proof is not overly difficult, but is a little bit cumbersome, as there are 
several cases to consider. The cases are based on what u and v are, and where 
they will be inserted. We do not want to break our line of thought here, and 
will therefore give the proof in Exercise 19. 

Now we are ready to prove Theorem 7.7. 


PROOF (of Theorem 7.7) Let 7 = ujug-+-Un, then a” = Untn—1-+ + U4. 
To alleviate notation, let ry, = rj, and let c,, = cj, and let us omit extraneous 
parentheses. We claim that 


CnCn—1 °° C1(0) = rire+++ Tn (0). (7.3) 


Proving (7.3) is sufficient as the left-hand side is clearly P(r)", and the right- 
hand side is P(x"). We prove (7.3) by induction on n, the initial case of n = 1 
being obvious. As our initial tableau is empty, it does not make any difference 
whether we row or column insert an entry into it. Therefore, 


rire ++ tn (O) = r1--+tn—16n (8) = Cnri +++ Tn-1(9), 


where the last equality holds because of Proposition 7.10. Applying the in- 
duction hypothesis by replacing rj ---Tn—1(@) by cn_1---c1(@), we get (7.3). 


The following theorem is perhaps even more interesting, and has a plethora 
of applications. It tells us what happens if we interchange the two tableaux 
that make up risk(7). 


THEOREM 7.11 
Let m € Sy, and risk(7) = (P,Q). Then we have risk(x~1) = (Q, P). 


This classic theorem is due to Marcel-Paul Schiitzenberger [282]. His orig- 
inal proof used induction. An elegant geometric proof that in fact proved a 
more general statement was given by X. Viennot in [320]. That proof can be 
found in English in [279]. 


Example 7.12 
Let 7 = 312, then 7~! = 231, and as it is easy to verify, risk(7) and risk(z~+) 
are shown in Figure 7.8. 
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P(231) Q(231) 
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FIGURE 7.8 


The images of 312 and its inverse, 231. 


In particular, 7 is an involution if and only if P(z) = Q(z). That is, the 
Robinson—Schensted—Knuth algorithm provides a bijection from the set of all 
n-involutions onto the set of all Standard Young Tableaux on n boxes. It is 
therefore easy to enumerate all these tableaux. 


COROLLARY 7.13 
The number of all Standard Young Tableauz on n boxes is equal to 


> fr= > & (26-1! 


|Fl=n i=0 


PROOF _ It is easy to see that the right-hand side is equal to the number 
of all involutions on [n]. Indeed, first choose the 27 entries that will be parts 
of 2-cycles in (J) ways, then take a fixed point-free involution on them. This 
latter can be done in (27 — 1)!! ways as we have seen in Corollary 3.55. | 


COROLLARY 7.14 
The equality 


2 2 
) fe = = exp (z+ 5) 

n! 2 
n>=0 


holds. 


The bijection g between involutions of length n and SYT on n boxes turns 
the number of fixed points of the involutions to a simple parameter of the 
corresponding SYT. See Problem Plus 2 for that result. The existence of g 
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also helps counting involutions avoiding monotone patterns, as illustrated by 
Problem Plus 11. 

There is an interesting, close connection between descents of permutations, 
and descents of the corresponding SYT. Recall that we say that 2 is a descent 
of a Standard Young Tableau Z if 7 appears in a row in Z that is strictly 
above the row in which i+ 1 appears in Z. 


THEOREM 7.15 
Let m © Sy, and leti € [n—1]. Theni is a descent of x if and only if i is a 
descent of Q(z). 


PROOF First let us assume that 7 € D(p), that is, 7; > 7:41. We need 
to show that the insertion of 7;,; results in the addition of a new box to the 
P-tableaux that is below the box resulting from the insertion of 7;. 

As 7 > Ti+1, we know that 7;41 gets inserted to the first row of P(z) 
weakly on the left of 7;. If the insertion of 7; ended in the first row, then we 
are done, as 741 will then have to displace an entry from the first row. 

In any case, the entry a, displaced from the first row by 7;+1 is smaller 
than the entry b; displaced from the first row by 7;. Therefore, even if the 
insertion of 7; ends in the second row, that of 7;,; has to go on to at least 
one more row. We can then repeat this argument for a, and b, instead of 
T+, and 7;, and the second row instead of the first row, and then iterate it 
for further rows. We then see that the insertion of 7,41 will always end below 
that of 7;. 

Now let us assume that ¢ ¢ D(p), that is, 7; < m41. Then reversing 
inequalities in the above argument shows that the insertion of 7:41 will end 
weakly above that of 7;. This implies that in Q(z), the entry 7 + 1 will be 
weakly above the entry 7, proving the second half of our claim. 


Dn 


7.2. Posets of Permutations 
7.2.1 Posets on S), 


There are various ways to define a partial order on the set of all n-permutations 
for a fixed n, or on the set of all finite permutations for that matter. The 
first two permutation posets that we mention, the Bruhat order and the weak 
Bruhat order, are ubiqitous in algebraic combinatorics as they can be gener- 
alized from S;, to a larger set of groups called Coxeter groups. The interested 
reader should consult [40] for these generalizations, as well as for further in- 
formation about these posets on permutations. 
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FIGURE 7.9 
The Bruhat orders on Sy and S3. 


7.2.1.1 Bruhat Order 


DEFINITION 7.16 Let P,, be the partially ordered set of alln-permutations 
in which p < q if p can be obtained from q by a series of operations, each of 
which interchanges the two entries of an inversion. Then P,, is called the 
Bruhat order on Si. 


For the sake of brevity, an operation that interchanges the two entries of 
an inversion will be called a reduction. The Bruhat order is sometimes called 
the strong Bruhat order for reasons that will become obvious shortly. 


Example 7.17 
Figure 7.9 shows the posets P2 and P3. Hl 


As the reader probably knows, in a poset, we say that y covers x if x < y, 
but there is no z so that x < z < y, or visually, when y is “immediately 
above” x. The reader should spend some time justifying some of the covering 
relations of P3, in order to become familiarized with this partial order. For 
instance, why does 231 cover 132? 

Recall that a poset is called graded if all of its maximal (non-extendible) 
chains have the same length, where the length of a chain is the number of its 
elements minus one. 
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PROPOSITION 7.18 
The Bruhat order P,, is a graded poset. 


PROOF  Weclaim that if y covers x in P,, then y has exactly one more 
inversion than x. This will obviously imply that all maximal chains of P,, 
have length (5). 

Let x < y, and let us assume that y covers x. By Definition 7.16, x < y 
means that x can be obtained from y by a series of reductions. However, as x 
is covered by y, all series of reductions that turn y into x must consist of one 
single reduction. Let that reduction be the transposition (y;y;), where i < j, 
while y; > y;, and let us assume that this reduction results in decreasing the 
number of inversions of y by more than one. That means that there is an index 
k so thati<k <j and y; > yx > y;. In that case, however, the permutation 
z = y(yiyrn) would satisfy « < z < y, contradicting the assumption that y 
covers &. 


In a finite graded poset, the rank of an element is the length of any max- 
imal chain ending at that element. It follows from the above proof that 
rankp, (p) = 1(p), where i(p) is the number of inversions of p € Sy. 

We are going to present a classic result that provides a characterization of 
the Bruhat order. To that end, we need an additional definition. Let p be an 
n-permutation. For each (a,b) € [n] x [n], we define 


p(a, b) = |{z € [a] so that p; > b}|. 


Example 7.19 

Let p = 31452, and let (a,b) = (3,2). Then p(a,b) = p(3,2) is the number 
of entries among the first three entries of p that are at least 2, that is to say, 
p3,2)=2. J 


Visually, we can think of p(a, b) as the number of dots in the a x (n—b+1) 
rectangle at the top left corner of the diagram of p, where the diagram of p is 
illustrated in Figure 7.10. 


THEOREM 7.20 
Let p and q be two n-permutations. Then p <p, q if and only if p(a,b) < 
g(a, b) for all (a,b) € [n] x [n]. 


See Theorem 2.1.5 in [40] for a proof. It is easy to prove that the condition 
is necessary, and the reader is asked to do so in Exercise 13. 
This characterization of the Bruhat order leads to the following theorem. 
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FIGURE 7.10 
The diagrams of p = 24153 and p—! = 31524. 
Diagram of p Diagram of q 
n—-b+1 A | n-A-b+1 n—b+1 B n—B-b+1 
b-1 a-A| A+b—a-1 b-1 a-B | B+b-a-l 
a n—a a n—-a 


FIGURE 7.11 
The number of dots in various rectangles. 


THEOREM 7.21 
Let p and q be two n-permutations. Then p <p, q if and only if p~! <p, q-t. 


PROOF _ As (p!)-! = p, it suffices to prove one implication. Using 
Theorem 7.20, it suffices to prove that if p(a,b) < q(a,) for all (a,b) € [nJ]?, 
then p~'(a,b) < q~'(a,) for all (a,b) € [n]?. Let us represent permutations 
with their diagrams as shown in Figure 7.10. It is obvious from the definition 
of inverse that the diagram of p~' is obtained from that of p by reflection 
through the x = y diagonal. 

As we mentioned right after Example 7.19, the number p(a, b) is just the 
number of dots in a rectangle of shape (n—b+1) xa in the top left corner of the 
diagram of p. So our condition means that no matter how large a rectangle 
we take in the top left corner of the diagram of p, the number of dots in 
that rectangle is never more than the number of dots in the corresponding 
rectangle in the diagram of q. 

All we need to prove is that the same inequality will hold for the rectangles 
that are in the bottom right corners of the two diagrams. Indeed, taking 
inverses turns these rectangles into top left corner rectangles. As each row and 
each column of our diagrams contains exactly one dot, it is easy to compute 
the number of dots in these rectangles, as shown in Figure 7.11. Clearly, if 
A< B,thenb—a-—1+A<b—a-—1+4B, and our theorem is proved. 
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FIGURE 7.12 
The weak Bruhat orders on Sj and S3. 


7.2.1.2 Weak Bruhat Order 


There is another partial ordering of permutations that is based on transposi- 
tions. 


DEFINITION 7.22 Let P’, be the partially ordered set of alln-permutations 
in which p < q if p can be obtained from q by a series of operations, each of 
which interchanges two consecutive entries that form an inversion. Then P’, 

is called the weak Bruhat order on S;,. 


Example 7.23 
Figure 7.12 shows the posets P} and Pj, = IJ 


It is obvious that y covers x in P’ if and only if 2 can be obtained from 
y by an adjacent transposition, that is, a reduction that interchanges two 
consecutive entries. This implies that the weak Bruhat order is graded, and 
rank(a) = i(x). 

As P’ is graded, all its maximal chains have the same length, namely (aye 
This means that in any maximal chain, the entries 7 and 7 are interchanged 
exactly once. 

Now that we know that the maximal chains of P’ are so similar to each 
other, we could ask how many such maximal chains exist. This question turns 
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FIGURE 7.13 
A balanced tableau. 


out to be remarkably interesting. It was first answered by Richard Stanley 
[294], who proved his own conjecture showing that the number of maximal 
chains of P’ is equal to the number of Standard Young Tableaux of shape 
(n —1,n —2,---,2,1). The proof in [294] uses symmetric functions, so the 
quest for a combinatorial proof has continued. At this point, we mention 
that a chain connecting permutation p to the identity permutation is called 
a reduced decomposition of p, so the task at hand is finding the number of 
reduced decompositions of n(n — 1)--+21. 

A year later Curtis Greene and Paul Edelman defined a new class of tableaux, 
called balanced tableaux, and then found a remarkably simple bijection between 
these tableaux of the same staircase shape and the maximal chains of P’. We 
will now give an overview of their results. See [151] for a short summary of 
all their results, and see [152] for the proofs of these results. 

Recall the definition of a hook from Chapter 6. For a box a in a Ferrers 
shape, let 2, denote the leg length of a, that is, the number of boxes that are 
in the same column as a, and are weakly below a. 


DEFINITION 7.24 Let F be a Ferrers shape that consists of m boxes. 
A balanced tableau of shape F' is a tableau whose boxes are bijectively filled 
with the entries of [m] so that for each box a, the content of a is the lath 
largest entry in the hook Hg. 


Note that there is some variation in the literature as to whether the content 
of a should be the ¢,th largest or (4th smallest entry in its own hook, but for 
staircase shapes, this causes no confusion. Indeed, the hooklength of a in such 
a shape is always 2@, —1, so the ,th largest entry of that hook is also its @,th 
smallest. 


Example 7.25 
Figure 7.13 shows a balanced tableau on nine boxes. 0 


THEOREM 7.26 
The number of maximal chains of the weak Bruhat order P! is equal to the 
number of balanced tableaux of shape (n — 1,n — 2,--+ ,2,1). 
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FIGURE 7.14 
The tableau f(C). 


PROOF Weare going to construct a bijection f from the set MC’, of all 
maximal chains of P’ to the set BalSt(n) of balanced tableaux of the staircase 
shape (n — 1,n — 2,---2,1). 

Let C € MC, be the maximal chain whose kth edge corresponds to the 
adjacent transposition interchanging the entries 7 and j, with i > 7. Then we 
define f(C) to be the tableau whose box in position (n + 1 — 7,7) contains 
the entry k. In other words, column 7 of f(C) describes the transpositions 
that moved j and a larger entry, while row n + 1 — 7 of f(C) describes the 
transpositions that moved 7 and a smaller entry. Here “describes” means “tells 
when it happened.” 


Example 7.27 

If n = 4, and c is the chain 1234, 2134, 2143, 2413, 4213, 4231, 4321, then 
f(C) is the balanced tableau shown in Figure 7.14. For instance, in the first 
step, c interchanges the entries 1 = 2 and 7 = 1, so the number 1 is placed 
into position (4+ 1 — 2,1) = (3,1) of f(C). 


We first show that f indeed maps into BalSt(n). Let us look at the box 
(rn +1-—%,7). We need to prove that its content k is indeed the (2 — 7)th 
largest entry in its hook Hy+1-4,;. 

The entries below the box (n+ 1 —i, 7) tell us when the interchanges (x, j) 
took place, where x < i. The entries on the right of the box (n + 1 — 4,7) 
tell us when the interchanges (i, y) took place, with y > j. Originally, i and 
j were at distance 7 — j from each other, so there had to be exactly i — 7 — 1 
interchanges of the above types before i and j could be interchanged. So the 
content & of the box (n + 1 — i,j) is the (i — j)th largest of its hook, as it 
should be. Therefore f(C) € BalSt(n). 

It is clear that f is an injection. Indeed, if C 4 C’, and the kth edge of C 
and C’ are different, then the position of k in f(C) will be different from that 
of k in f(C’). 

We still need to prove that f is a surjection, that is, that for all B ¢€ 
BalSt(n), there exists a C € MC), so that f(C) = B. Take B € BalSt(n), 
and try to find its preimage under f. Then B specifies an order in which we 
should carry out the i) transpositions on the decreasing permutation. What 
we have to show is that the balanced property of B assures that in every step, 
we will be asked to carry out an adjacent transposition. 
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In order to prove this statement, we need the following, somewhat surprising 
characterization of balanced tableaux, given by Greene and Edelman. It shows 
that in fact, staircase shaped balanced tableaux are even more “balanced” 
than we might think. Let c(t, 7) = cg(t,7) denote the content of box (i, 7) of 
a given tableau B. We will omit B when there is no danger of confusion. 


LEMMA 7.28 

[152] Let B be a tableau of staircase shape (n —1,n —2,--- ,1) whose boxes 
are biectively filled with the elements of [(3)I- Then B is balanced if and only 
if, for all (i, 7) satisfying it j <n, and for all s > i, exactly one of c(s,7) 
and c(t,n — s +1) is larger than c(i, 7). 


To remember the two boxes whose content brackets c(i, 7), note that in the 
first one, the second coordinate is fixed, the first is changed to s, in the second 
one, the first coordinate is fixed, the second is changed to n — s+ 1. 


PROOF The “if” part is not surprising. Indeed, H;,; — (i,j) can be 
partitioned into disjoint unions of pairs {c(s, 7), c(t,n — s+ 1)}, and if c(2, 7) 
is larger than exactly one element from each pair, then it is larger than half 
of the entries in H;,; — (2,7), and so B is balanced. 

It is surprising, however, that this seemingly stronger condition is also nec- 
essary, that is, if B is balanced, then this condition has to hold. Let us assume 
that this is not true, and let us further assume, without loss of generality, that 
there exists s > i so that c(i,j) > c(s,j), and also, c(i, 7) > c(i,n — s +1). 
Then, by the balanced property of B, there exists another index t so that 
c(i, 7) < c(t, 7), and also, c(i, j) < c(t,n—t+1). Let us assume, again without 
loss of generality, that t > s. 

Let us now assume that H;,; is a minimal counterexample to our statement. 
(It is easy to check that in that case, we must have h;,; > 5.) The reader is 
invited to follow our argument in Figure 7.15. For brevity, in this figure we 
set x’ = n—a-+1, and also, we marked entries known to be larger than c(i, 7) 
by black dots, and entries known to be smaller than c(i, 7) by gray dots. 

As we assumed that H;,; was a minimal counterexample, our statement 
must hold for H,,;, and Hj. Therefore, as our statement holds for H,;, we 
must have 

c(s,t’) < e(s,9) < c(é, 9). 


On the other hand, as our statement holds for H(i, t’), we must have 
c(s,t') > c(t, #) > c(, 9). 


As the last two chains of inequalities clearly contradict each other, our lemma 
is proved. 
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FIGURE 7.15 
The entry c(s,t’) leads to a contradiction. 


Now we can return to the proof of the surjectivity of f. Let B € BalSt(n), 
and let us start building up a maximal chain C by decoding B. That is, in step 
m, let us interchange the entries i and 7, where m = c(n—i+1,7) in B. We 
must show that each step will actually define an adjacent transposition. Let 
us assume that this is not the case, and let k be a minimal counterexample. 
That is, let us assume that step k asks us to interchange x and y, so that 
xz <y, but x and y are not in consecutive positions. 

First we show that if « < z < y, then z cannot still be between x and 
y. Indeed, look at Hy+1~-y,2. This hook has k written in its peak, and its 
length is 2(y— x) — 1. As B is balanced, there are y — x — 1 entries in 
An+1-y,2 that are less than k. This means that before step k, there have 
been y — x — 1 interchanges of the types (x, z), with x < z, and (z,y), with 
z<y. So no z € (a,y) could be still located between x and y. Now let 
vu<a<_y, and let us assume that v has been interchanged with x at some 
point before step k. This means that c(n + 1—2,v) < k. However, by 
Lemma 7.28, c(n+1—2,v) has to be larger than exactly one of c(n+1-—y,v) 
and c(n+1—2,y) =k. Because of the previous sentence, the only way for 
this to happen is c(n +1—y,v) < cn +1-—2,v) < k. So in particular, 
c(n +1—y,v) < k, meaning that y and v were also interchanged before step 
k, so v is not located between x and y. 

Finally, an analogous argument shows that if « < y < u, and u and y were 
interchanged before step k, then so were u and x. Therefore, no element can 
be located between x and y, so x and y are adjacent as claimed. Consequently, 
the chain C' can always be built up so that f(c) = B, and our proof is complete. 


We have mentioned that Stanley [294] has proved that the number of all 
maximal chains in the weak Bruhat order of S;, is the number of all Standard 
Young Tableaux of shape (n — 1,n—2,--- ,2,1). Now we have seen the proof 
of Edelman and Greene showing that this number is also equal to the number 
of balanced tableaux of shape (n — 1,n — 2,--- ,2,1). This certainly means 
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that there are as many balanced tableaux of this shape as SYT of this shape. 
It turns out that a much more general statement is true. 


THEOREM 7.29 
[152] Let F be any Ferrers shape, and let b" be the number of all balanced 
tableaux of shape F. Then 

bf = fF. 


In other words, there are as many balanced tableaux of any given shape as 
there are SYT of that same shape. See [152] for a proof of this theorem. The 
proof follows from some sophisticated bijections. There are some sporadic 
cases in which the bijection is simple, but in the general case it is not. 

Theorem 7.29 shows that the number of balanced tableaux of a given shape 
is also given by the hooklength formula. It would be interesting to find a 
probabilistic proof of this fact. 


7.2.2 Posets on Pattern-Avoiding Permutations 


Let PA be the partially ordered set of 132-avoiding n-permutations ordered 
by strict containment of the descent sets. That is, in P4 we have p < q if 
D(p) c D(q). 

It is then clear that P“ is ranked, and we know from Problem Plus 1 of 
Chapter 4 that there are 4(7) (eu) elements of rank k in P4. In particular, 
this means that P4 has as many elements of rank i as of rank n — 1 —3, in 
other words, PA is rank-symmetric. 

There is a much deeper notion of symmetry in posets. 


DEFINITION 7.30 We say that a poset P is self-dual if it has an anti- 
automorphism, that is, if there exists a bijection f : P > P so that p<pq if 
and only if f(a) < f(p): 


A bijection f having the property required in the above definition is often 
called order-reversing because of what it does. 


Example 7.31 
For any positive integer n, the Boolean algebra B,, of all subsets of [n], ordered 
by containment, is self-dual. 


SOLUTION Let f be the bijection that maps each subset of [n] to its 
complement. Then f is clearly order-reversing, proving our claim. 
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THEOREM 7.32 
The poset PA is self-dual. 


PROOF (of Theorem 7.32) Let p € P4. Recall Exercise 27 of Chapter 4. 
Define f(p) to be the 132-avoiding permutation whose unlabeled binary tree is 
obtained from the unlabeled binary tree T(p) of p by reflecting T(p) through 
a vertical axis. By part (b) of the mentioned exercise, this reflection will turn 
left edges into right edges, and so ascents into descents, and vice versa. In 
particular, the vertex that was in the ith position from the left will now be 
in the ith position from the right. Therefore, by part (c) of the mentioned 
exercise, if pj > pj41, then we will have f(pn_i) < f(pPn4i_i). So i € D(p) if 
and only ifn —i ¢ D(f(p)) for i € [n— 1]. 

In other words, the descent set of f(p) is precisely the reverse complement 
of the descent set of p, implying that f is order-reversing. 


Let Qé be the poset of 321-avoiding n-permutations ordered by strict con- 
tainment of the set of excedances. 


PROPOSITION 7.33 
The posets PA and QA are isomorphic. 


PROOF _ For each S C [n — 1], let £3?'(S) be the set of 321-avoiding 
n-permutations with excedance set S C [n — 1]. 

Let also D1°?(a(S)) be the set of 132-avoiding n-permutations with descent 
set equal to a(S), the reverse-complement of S. 

We construct a bijection s: £321($) > D}9?(a(S)) illustrated by Example 
7.34. If p € E??'(S), then, as seen earlier in the definition of @, the entries 
p; with 7 ¢ S form an increasing subsequence. This, and the definition of 
excedance imply that p; is a right-to-left minimum (that is, smaller than all 
entries on its right) if and only if 7 ¢ Exc(p) = S. 

Now let p! = pnpn—1-+- pi be the reverse of p. Note that p’ is a 123-avoiding 
permutation that has a left-to-right minimum at position 7 < n exactly if 
n+1-ié€S. 

Recall from the proof of Lemma 4.4 that there is exactly one 132-avoiding 
permutation p” which has the same set of left-to-right minima and has them 
at these same positions. Namely, p” is obtained by keeping the left-to-right 
minima of p’ fixed, and successively placing in the remaining positions, from 
left to right, the smallest available element that does not alter the left-to- 
right minima. We set s(p) = p”. From the proof of Theorem 7.32 we see 
that i € D(p”) if and only if n—7 € S for i € [n — 1], in other words, when 
i € a(S), and so p” belongs indeed to D}8?(a(S)). 

It is easy to see that s is invertible. Clearly, p’ can be recovered from p” as 
the only 123-avoiding permutation whose left-to-right minima have the same 
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values and positions as those of p”. (All entries that are not left-to-right 
minima are to be written in decreasing order.) Then p can be recovered as 
the reverse of p’. 

The bijections s: E3?1($) + D}3?(a(S)) for all choices of S C [n — 1] 
produce an order-reversing bijection from QA to PA. But PA is self-dual, so 
the proof is complete. 


Example 7.34 

Take p = 34162951078 € Ef2!(S) for S = {1, 2,4, 6,8}. Then its reversal 
p =87105926143 has left-to-right minima 8, 7, 5, 2, 1 in positions 
1, 2, 4, 6, 8. We obtain s(p) = p’ = 879562314 10, a permutation in 
p13?({1,3,5,8}). 


7.2.3 Infinite Poset of Permutations 


Let P be the poset of all finite permutations ordered by pattern containment. 
That is, in this poset, p < q if and only if p is contained in q as a pattern. 
This means that permutation classes defined in Chapter 4 are precisely the 
ideals of P. 

We have seen in Chapters 4 and 5 that if we want to find permutations that 
avoid all of the patterns contained in a given set S, then our task is getting 
progressively harder if new elements are being added to S. It is therefore 
reasonable to ask whether this task will eventually become impossible. That 
is, let N be an arbitrary positive integer. Is it possible to find N permutations 
so that none of them contains any other as a pattern? Or, even more strongly, 
is it possible to find an infinite antichain in the poset P? 

This question was attacked, and the affirmative answer discovered and re- 
discovered several times, during the last third of the twentieth century. Here 
we present a construction that may be chronologically the first. The construc- 
tion (without proof of the antichain property) was published by Tarjan [310] 
in 1972. 


THEOREM 7.35 
The poset P contains an infinite antichain. 


The above result could be reformulated by saying that P is not a well-quasi- 
ordering. 
PROOF _ Let n> 2, and let 
Dn = 2(4n—1)416385---(4n — 2) (4n — 5) (4n) (4n — 3). 


In other words, p,, has 4n entries, and consists of two parts, the increasing 
subsequence 246---(4n) in the odd positions, and the odd entries in the even 
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positions in increasing order, except for 4n — 1, which is moved up into the 
second position. We claim that the p, form an infinite antichain. 

Assume the contrary, that is, that py <p, for some k < n. Which entries 
of p, could play the roles of the entries of p,? The role of 4k — 1 has to be 
played by 4n — 1; otherwise, we could only find at most two smaller entries on 
its right. Therefore, the entry 2 of p, must play the role of the entry 2 of px. 
This, however, totally ties our hand in making the remaining 4k — 2 selections. 
Indeed, the entry 1 of p,, must be chosen to play the role of 1 as that is the 
only entry smaller than 2; therefore, the entry 4 of p, has to be chosen to 
play the role of 4 as that is the only entry not yet selected that precedes 1. 
These forced selections continue, and we have to choose the leftmost 44 — 2 
entries of p,, to play the roles of the first 4k — 2 entries of p,,. Then we must 
choose 4n to be the next-to-last entry as that is the only entry larger than 
4n — 1, forcing us to choose 4n — 3 to play the role of the last entry of pz. 
However, this last entry is too large. Indeed, it is larger than the entry 4k — 2 
that we choose when we selected the (4k — 3)th entry of our purported copy 
of p,. This is a contradiction, as in pz the last entry is smaller than the entry 
in position 4k — 3. 


(I 


7.3 Simplicial Complexes of Permutations 


We have defined simplicial complexes in Problem Plus 1 of Chapter 1, but we 
repeat that definition for easy reference. 


DEFINITION 7.36 A simplicial complex A is a family of subsets of an 
underlying set S so that if F © A andGcCF, thnGeA. 


In other words, a simplicial complex is an ideal of the Boolean algebra with 
underlying set S. The sets that belong to the collection A are called the faces 
of A. If S € A has i elements, then we call S an (i — 1)-dimensional face. 
The dimension of A is, by definition, the dimension of its maximal faces. 


Example 7.37 
Let P be a finite partially ordered set. Then the collection of all chains in P 
forms a simplicial complex, called the chain complez of P. 


Indeed, every induced subposet of a chain is a chain. 

If we can prove that certain objects, say, n-permutations with k descents, 
are in bijection with k-element faces of a given simplicial complex, then 
that can have additional algebraic significance. In fact, additional algebraic 
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interpretations can be found for the numbers enumerating our objects. The 
algebraically inclined reader is invited to consult [95] for details. 


7.3.1 Simplicial Complex of Restricted Permutations 


As we have mentioned, and in some cases, shown, there are many objects 
enumerated by the Catalan numbers. Among all of these, we now choose 231- 
avoiding permutations and show how they form a simplicial complex. The 
reason for this choice is that 231-avoiding permutations fit well in a more gen- 
eral class of permutations, called t-stack-sortable permutations, to which we 
will return in the next chapter. The following theorem explains what we mean 
when we say that the set of these permutations form a simplicial complex. 


THEOREM 7.38 

There exists a simplicial complex (with an underlying set of C) elements) 
whose (k—1)-dimensional faces, that is, k-element faces, are in bijection with 
231-avoiding n-permutations having k ascents. 


PROOF We have seen in Exercise 17 of Chapter 4 that 231-avoiding 
n-permutations are in bijection with northeastern lattice paths from (0,0) 
to (n,n) that never go above the main diagonal. We have also seen in that 
same exercise that this bijection turns the ascents of the permutations into 
north-to-east turns of lattice paths. 

Now let A be the simplicial complex of all sets S of points for which there 
exists a northeastern lattice path r so that the set of all north-to-east turns of 
r is equal to S. (Of course, we still have to show that A is indeed a simplicial 
complex.) It is clear that given S, we can recover r. So the faces of this sim- 
plicial complex A are indeed in bijection with 231-avoiding n-permutations. 
The previous paragraph shows why (k — 1)-dimensional faces correspond to 
permutations with k ascents. 

Finally, we prove that A is indeed a simplicial complex. Let S € A. Then 
there exists a subdiagonal northeastern lattice path r so that the set of north- 
to-east turns of r is equal to S. That is possible if and only if the points of 
S form a chain in N? (in the natural cordinate-wise ordering), are between 
certain limits, and have all different horizontal and vertical coordinates. How- 
ever, if that is the case, then that must be true for all subsets T’ C S, implying 
that there is a northeastern lattice path t whose set of north-to-east turns is 
T. That means T € A, and the proof is complete. 


Example 7.39 
Let n = 4, and let S = {(2,1), (3,2)}. Then the northeastern lattice paths 
whose set of north-to-east turns is contained in S are shown in Figure 7.16. 
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FIGURE 7.16 
Each subset of S defines a lattice path. 


7.3.2 Simplicial Complex of All n-Permutations 


There are several ways to define a simplicial complex whose k-element faces 
are in bijection with n-permutations having & descents. One of these was 
presented in Theorem 1.27, and others can be found in [189] or [153]. Here 
we present such a simplicial complex based on the bijective representation of 
permutations by labeled lattice paths. We have discussed this representation 
in the proof of Theorem 1.27, but for easy reference, we include it here. 

Let p = pip2:::Pn be an n-permutation having & descents, and let us say 
that D(p) = {d1,d2,---,d,}. Then it is not difficult to prove that we can 
bijectively represent p by a northeastern lattice path F(p) consisting of k 
vertical and n — k horizontal edges, where the edges are labeled according to 
the following rules. 

Let the edges of F'(p) be denoted a1, a2,--- ,a@,, and let e; be the label of 
a;. Then the following has to hold. 


(i) The edge ay is horizontal and e; = 1, 


(ii) If the edges a; and a;41 are both vertical, or both horizontal, then 
€; = Ci41, 


(iii) If a; and aj41 are perpendicular to each other, then e; + e;41 <i+1. 


The set of labeled lattice paths of length n satisfying these conditions is de- 
noted P(n). 
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F(p) 


FIGURE 7.17 
Decomposing F'(p) into (F%1), Fray). 


We are going to decompose F'(p) into a k-tuple of northeastern lattice paths, 
(Fay, Fi2),:++ »F(e)), each of which will consist of one vertical step and n—1 
horizontal steps. The unique vertical step of F(;) will be in the same position 
as the ith vertical step of F'(p). In other words, the unique vertical edge of 
Fy) will be its (d; + 1)st edge, corresponding to the ith descent of p. We still 
have to specify the labels of the edges of Fi). Let e;,; be the label of the jth 
edge of Fy;). We then set 


Cig = ej if d; +1 <j < di4i, 
lif djz, +1 <j. 


It is straightforward to check that this is a valid definition, that is, rules 
(i)-(iii) are satisfied. 


Example 7.40 
Figure 7.17 shows how we decompose F'(p) if p = 612435. 


Now let A be the family of k-tuples (Fi), Fi2),--+ , F(x) that can be ob- 
tained from n-permutations the way described above, for some k. Note that 
instead of saying “k-tuples,” we might as well say “sets” as the order of the 
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F(;) within each such set is completely determined by the position of the single 
vertical edge in each F%,). 


THEOREM 7.41 
The collection of sets A defined in the previous paragraph is a simplicial com- 
plex. 


PROOF The proof is very similar to that of Theorem 7.38. Let F’ = 
(Fuy, Fi),:++ ,Fix)) be a k-tuple of lattice paths from P(n) having one single 
vertical edge each. Define the numbers 6; so that the vertical edge of FY) is 
the bjth edge of Fy,). Then F’ € A if and only if by < by < +--+ < by. If F’ 
has this property, then obviously so do all its subwords (subsets), so F”’ € A 
implies F” € A, forall FP” c F’. 


(I 


Exercises 
1. (-) Prove that I, (123--+k) < (k—1)”. 


2. Noncrossing partitions of [n] are defined in Exercise 24. Find a bijection 
from the set of 321-avoiding permutations with k excedances to the set 
of noncrossing partitions of [n] having k + 1 blocks. 


3. Prove, without the use of the hooklength formula, that the number of 
Standard Young Tableaux of shape 2 x n is Ch. 


4. What is the average number of descents of a 2 x n Standard Young 
Tableau? 


5. Let a be a permutation of length 80 that avoids the pattern 12---9, 
and let us assume that P(7) has ten rows. How long is the fifth row of 
P(x)? 


6. Find a formula for the number of northeastern lattice paths from (0, 0) 
to (n,n) that never go above the main diagonal and are symmetric about 
the diagonal x + y =n. 


7. Prove that In(123) = (j,"79))- 
8. Prove that [,,(123---k) is P-recursive in n. 


9. Let Dx,-(n) denote the number of n-permutations p in which the longest 
increasing subsequences have k elements and for which r is the largest 
natural number so that there exist r disjoint increasing subsequences of 
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10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


18. 


19. 
20. 
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maximum size in p. Prove that then D;,,,(n) is a P-recursive sequence 
of n. 


Let 7 be a permutation so that the first row of P(7) is of length a, and 
the second row of P(7) is of length 6. Is it true that 7 has two disjoint 
increasing subsequences s; and sz so that s; has length a and sg has 
length b? 


(—) Prove that 
ee eae 
F’ 
where F” ranges over the set of Ferrers shapes that can be obtained from 
F by omitting a box (which is necessarily an inner corner). 


Let 7 be any permutation other than the increasing or decreasing one. 
Find a simple way to create another permutation o so that P(7) = P(c). 


Prove part of Theorem 7.20 by showing that if p(a,b) > q(a, b) for some 
(a, b) € [n] x [n], then p < q cannot hold. 


For what positive integers n is the number of involutions of length n 
even? 


Is there an infinite antichain in the poset P of finite permutations or- 
dered by pattern containment that consists of 123-avoiding permuta- 
tions? 


(—) Keeping in mind that the reverse or the complement of an involution 
is not necessarily an involution, prove that nevertheless, for all positive 
integers n, we have 


I, (123+++k) = In(k+ +321). 


We can view a permutation p as a poset P, as follows. The elements 
of P, are the entries of p, and x <p y if (x,y) is a non-inversion of p. 
Prove that P, and P,-1 are isomorphic. 


Let k be any positive integer. For some n, find k different n-permutations 
p; so that all the P,, are isomorphic to one another. 


Prove Proposition 7.10. 


Let us consider all n! vectors that are obtained by permuting the coor- 
dinates of the n-dimensional vector 


1 
2 
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21. 


22. 


23. 


24. 


25. 


26. 


Denote by 7, the convex hull of all of these vectors in R”, that is, the 
smallest convex set that contains all of these vectors. It is clear that 
Tp is a polyhedron, and therefore, it is often called the permutahedron. 
How many edges does 7, have? 


Prove that 
3 = @ a AG) 
<i n+1 k+1 ntl 
(+) Prove that the number of dissections of a convex (n + 2)-gon by 


d nonintersecting diagonals is equal to the number of Standard Young 
Tableaux of shape (d+ 1,d+1,1,1,---,1), where the number of rows 
of length 1 isn —1—-d. 


Complete the proof of Theorem 7.4 by showing that the hooklength 
formula indeed implies (7.1). 


Recall the definition of a crossing in a set partition from Problem Plus 
9 of Chapter 5. Let us call a partition noncrossing if it has no crossings. 


(a) Define a bijection from the set of noncrossing partitions of [n] hav- 
ing k blocks to the set of 132-avoiding n-permutations having k—1 
descents. 


(b) Define the partially ordered set NC(n) of noncrossing partitions 
of [n] by refinement. That is, t <no(n) 7’ if all blocks of 7’ are 
unions of blocks of 7. Compare this poset to the poset P4 defined 
in Section 7.2.2. 


Recall that a partially ordered set P is called a lattice if for any two 
elements x,y € P, the set 


{z€ Pla < z and y < z} 
has a (unique) minimum element x V y, and the set 
{ue Plz >uand y > u} 


has a (unique) maximum element x A y. 
Is the weak Bruhat order P’, a lattice? 
Prove that there exists a bijection f : P’ > P’ so that «A f(x) = 


123---n and x V f(%) = n(n — 1)---21. See the previous exercise for 
the relevant definitions. 
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35. 
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37. 
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A finite lattice L is called complemented if for any element x € L, there 
exists a unique element y € L so that x\ y=0 and «Vy = 1, where 0 
denotes the minimum element of L, and 1 denotes the maximum element 
of L. 


Is P’ a complemented lattice? 


Let I, be the induced subposet of the (strong) Bruhat order P,, whose 
elements are the involutions of length n. Is I, a lattice? 


Let I, be defined as in the previous exercise. Is I, self-dual? 


The dimension of the poset P is the smallest positive integer d so that 
P is the intersection of d total orderings. What is the dimension of the 
poset P, defined in Exercise 17? 


(+) Find an asymptotic formula for the number of 2-dimensional posets 
on n labeled elements. 


We have defined the permutahedron in Exercise 20. Clearly, 7, is in 


fact an (n — 1)-dimensional polyhedron, as all its vertices lie within the 


hyperplane 30", 2; = eae So in particular, 74 is three-dimensional. 


Do all faces of 74 have the same number of edges? 


Let p be a randomly selected involution of length n. Let i € [n — 1]. 
What is the probability that i is a descent of p? 


Decide if the following statements are true or false. In all three state- 
ments, p and q are two n-permutations, and their respective sets of 
inversions are I(p) and I(q). 


(a) If I(p) < I(q), then p < q in the Bruhat order. 
(b) If p < q in the Bruhat order, then I(p) < I(q). 
(c) If p < q in the weak Bruhat order, then I(p) < I(q). 


Let a, be the number of involutions of length n. Prove that 
n! < a2 < nlp(n), 
where p(n) is the number of partitions of the integer n. 
Let a, denote the number of involutions of length n. Prove that 
(*") nl= 3 & Cty isa es: (7.4) 
n mene 


Let us call two n-permutations p = p,p2---py and q = q1q2°::Gn col- 
liding if there is at least one index i so that |p; — q;| = 1. Let S be a set 
of n-permutations so that each pair of elements of S is colliding. Prove 
that S has at most bias ) elements. 
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38. 


39. 


40. 


Let p(n) be the size of the largest set of n-permutations in which every 
pair of elements is colliding. Prove that p(n) > p(n — 1) + p(n - 2). 


Prove that for all positive integers n, the equality Av,(231,312) = 
I,(231, 312) holds. 


Prove that there are no pairs (p,q) of patterns of length three or longer, 
other than those in the preceding exercise so that Av, (p,q) = In(p, ¢g) 
holds for all p and gq. 


DT 


Problems Plus 


dr, 


Let Dz(n) be the number of n-permutations in which the longest in- 
creasing subsequences have k elements and they all have at least one 
element in common. Prove that D;,(n) is a P-recursive function of n. 


. We have seen in Theorem 7.11 that the Robinson—Schensted—Knuth 


correspondence naturally defines a bijection between inversions of length 
n and SYT of size n. Let f be this bijection, and assume that we are 
told the shape of f(p) (so not the content of each box). How can we 
figure out the number of fixed points of p from this information? 


. What is the number of 321-avoiding fixed point-free involutions of length 


2n? 


. What is the number of 123-avoiding fixed point-free involutions of length 


2n? 


. Prove that the number of 123-avoiding involutions of length 2n+1 having 


exactly one fixed point is 


72123) = (°" : '). 


n 


. Let rh” (q) denote the number of g-avoiding involutions of length n hav- 


ing exactly k fixed points. Prove that 


BT ncaa) for n+ k even, 


1) (321) = 1) (132) = 1) (213) = 
0 for n+k odd. 


. Let P be a poset having n elements, and let a; denote the length of its 


longest chain. Let ag be defined so that the largest number of elements 
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12. 


13. 
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that the union of two chains can contain in P is a, + ag. Similarly, for 
i > 2, let a; be defined so that the largest number of elements that the 
union of 7 chains can contain in P is a; -+a2+---+a,;. Continue defining 
the a; as long as they are positive. 


(a) Prove that a, > az >--- > ax, where ax is the last a; that has been 

defined (in other words, See a; =n). Therefore, there exists a 
Ferrers shape F' = (a1, d2,:-- , ax). Let 6; be the length of column i 
of F’. Prove that for any positive integer j, the sum bj +b2+---+0; 
is equal to the largest number of elements that can be covered by 


j antichains in P. 


(b) Why is the result of part (a) a generalization of Theorem 7.5? 


. Let us try to generalize the result of the previous Problem Plus as fol- 


lows. Let G be a graph on n vertices, and let a; be defined so that 
a, + a2+---+ a, is equal to the largest number of vertices that can be 
contained in the union of 7 cliques (complete subgraphs) in G. Again, 
define the a; as long as they are positive. 


(a) Is it true that a1 > ag >--- > ax? 

(b) Assuming that the answer to the question of part (a) is yes, define 
b; as in part (b) of the previous Problem Plus. Is it true that the 
sum 6; + bo +--+ + 6; is equal to the largest number of elements 
that can be contained in j anticliques (independent sets of points, 
in other words, empty subgraphs) in G? 


. For what class of graphs will statements (a) and (b) of the previous 


Problem Plus follow directly from the result of Theorem 7.5 ? 


. Find a formula for [,,(1234). 
11. 


(a) Find a formula for [,,(12345). 
(b) Find a formula for [,,(123456). 


Prove that for all positive integers and n, and for all even positive inte- 
gers k, we have 


(*") Av,(12---k+1) = », i (SF Roe we Fa): 


Let z,(m) denote the number of fixed point-free involutions of length m 
with no decreasing subsequences of more than k elements. Prove that 
for all positive integers n, and for all odd positive integers k the equality 


> (”") xr(r)tn(2n — 1) = ey |) [,(12---k + 1)Ion-r(12-+++k +1) 


r r=0 
(7.5) 
holds. 
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14. Prove that for all positive integers n, the equality 


2n 


SC) (*") T,.(1234) Ion (1234) = Inn (12345) (7.6) 
r=0 


holds. 


15. Let F'(2n) be the induced subposet of the Bruhat order P2,, whose ele- 
ments are fixed point-free involutions of length 2n. 


The rank-generating function of a finite graded poset P is the polynomial 
Wp(x) = ae r,x', where r; is the number of elements of P that are 
of rank 7. Prove that 


Wren) = [1][3]---[2n — 1]. 


16. Find an asymptotic formula for the number of isomorphism classes of 
two-dimensional posets on n elements. Here [m] = 1+q+---+q™7!. 


17. Let k > 2 be an integer, and let us say that two n-permutations p = 
Pip2°**Pn and q = qiq2°-:Qn are k-different if there exists an index i 
so that pj # qi, while 1 < pi,q; < k. Let K(k) be the largest integer 
for which there exists a K(k)-element set of n-permutations that are 
pairwise k-different. Prove that K(k) > (k+1)!/2. 


18. Recall that in Exercise 52 of Chapter 4, we defined u,,(q) to be the num- 
ber of permutations that avoid g and have a unique longest increasing 
subsequence, or ULIS. Prove that limpsoo (tn(321))!/" = 4. 


a 


Solutions to Problems Plus 


1. We claim that our condition that all maximum-length increasing subse- 
quences have a common element is equivalent to the seemingly weaker 
condition that any two maximum-length increasing subsequences inter- 
sect. If we can prove this, then our problem will be reduced to the 
special case r = 1 of Exercise 9. 


To prove our claim, let us assume that p is a permutation in which 
any two increasing subsequences of maximum length k intersect. We 
construct a directed graph G, associated to p. The vertices of Gp are 
the entries of p and there is an edge from the entry 7 to the entry 7 if 
and only if 2 < 7 and 27 is on the left of 7. So an increasing subsequence 
of length k in p corresponds to a directed path of length k in G,. Now 
let us remove all edges not in any maximum-length-path from G,, and 


342 


Combinatorics of Permutations, Third Edition 


add a “source” s and a “sink” ¢ to get the graph G’,. That is, s and 
t are vertices so that s has indegree zero, and there is an edge from s 
to all left-to-right minima of p, while t has outdegree zero and there 
is an edge to ¢ from all right-to-left maxima of p. So each increasing 
subsequence of size k corresponds to an s + t path of maximum size in 
a natural way. Now suppose these directed paths of maximum length do 
not have a vertex in common. Then we can delete any vertex v and still 
have an s + t path in G‘,. In other words, Gi, is 2 — (s,t)-connected, 
which implies, by the famous theorem of Menger (see for example [246]) 
that there are at least two vertex-disjoint s > t paths in G’,. This is 
equivalent to saying that there are two increasing subsequences of size k 
in p that are disjoint, which is a contradiction and the proof is complete. 


The number of fixed points of p is equal to the number of columns of 
odd length in f(p). See [32] for a proof of this fact, and for a reference 
for the original paper of Schiitzenberger where the result was proved. 


We have seen in the previous Problem Plus that the number of fixed 
points of an involution equals the number of odd columns of its P- 
tableau. Therefore, a fixed point-free involution has no odd columns. 
If, in addition, such an involution avoids 321, then its P-tableau cannot 
have columns longer than 2. This implies the shape of this P-tableau 
must be 2 x n. As an involution is completely determined by its P- 
tableau, we conclude from Exercise 3 that the number of such SYT is 
Ch. 


Let us first try to proceed as in the previous Problem Plus. In any case, 
the rows of the P-tableau of such an involution must be of length at most 
two, so the P-tableau has two columns. Because of the fixed point-free 
criterion, odd columns are not allowed. So (except for the trivial, one- 
column case), the P-tableau will have two columns, of length 2(n — k), 
and 2k, where k < n/2. It is then routine to compute by the hooklength 
formula that the number of these tableaux, and therefore, the number 
of fixed point-free 123-avoiding involutions of length 2n is 


3 ee On — 4k +1 
ar 2k} 2n—-—2k+1 
This is not a particularly simple formula. 


However, it has recently been proved in [146] that in fact, the number of 
these involutions is Ce The proof uses bijections to certain lattice 
paths called Dyck paths. 


It follows from the solution of Exercise 7 that Ion41(q) = eng As 


a 123-avoiding involution of odd length has to have exactly one fixed 


point, the result follows. 
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FIGURE 7.18 
A counterexample. 


6. It is clear that 1 (132) = i (213) for any n and k as 231 is the reverse 
complement of 132, and the reverse complement of an involution is an 
involution. The enumeration formula for 132 is proved in [205], and 
the bijection between the sets enumerated by 1) (132) and i (321) is 
given in [146]. 


7. (a) 


— 
& 


This is the famous Greene—Kleitman—Fomin theorem, which has 
several proofs. The two earliest ones are [200] and [176]. A differ- 
ent proof, using the concept of orthogonal families of chains and 
antichains, was given in [179]. 

For any permutation p, we can take its permutation poset P,. The 
chains of P, will be in bijection with the increasing subsequences of 
p, and the antichains of P, will be in bijection with the decreasing 
subsequences of p. Note that in this special case, the proof of our 
claim follows from Theorem 7.7. 


No, this is not true. See Figure 7.18 for a counterexample. It is 
easy to see that for the graph shown in that figure, we have a; = 3, 
a2 = 1, and ag = 2. 

This is not true either. See Figure 7.19 for a counterexample. One 
checks easily that a, = ag = 2, and a3 = 1. This would have to 
imply 6; = 3, but there is no anticlique of size three in this graph. 


9. The class of graphs we are looking for is that of comparability graphs 
of posets. If P is a poset, then its comparability graph G(P) is the 
graph whose vertex set is the set of elements of P, and two vertices are 


344 Combinatorics of Permutations, Third Edition 
So 
4 
So 
e ° 
\ / 
\ / 
\ 
\ / 
\ / 
i if 
\ / 
\ / 
\ / 
@ @ 
FIGURE 7.19 


A counterexample. 


10. 


11. 


connected by an edge if the corresponding elements of P are compara- 
ble. Then the cliques of G(P) correspond to the chains of P, and the 
anticliques of G(P) correspond to the antichains of P. 


The numbers J,,(1234) are equal to the Motzkin numbers, that is, 


n/2 


I, (1234) = me @ 


This result was first proved by Amitaj Regev in [272], who used sym- 
metric functions in his argument. A simpler proof is given in [297], 
Exercise 7.16.b, but that proof still uses symmetric functions. In recent 
years, there are a plethora of results on the subject, that together yield 
that [,(1234) = M,, and do not use symmetric functions. In fact, it is 
known that 

M,, = [,(2143) = I,(1243) = I, (1234). 


The first of the above three equalities was bijectively proved in [207], 
the second one was bijectively proved in [206], and the third one follows 
from a result proved in [220]. That result is [,(12@q) = In(21@4q), for 
any pattern g. Therefore, 


Ty, (1243) = In (2134) = I, (1234), 


where the first equality is true because if p is an involution, then so is 
the reverse complement of p. 

We point out that [220] contains the even stronger result that [,,(123 8 
q) = I,(321 © q) for any pattern g. It would be interesting to find a 
more direct, combinatorial proof. 


(a) The number J,,(12345) is in fact the number of Standard Young 
Tableaux on n boxes having at most four columns. It is proved in 
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12. 


13. 


14. 


[197] that 
C2 if n = 2k is even, 
I,(12345) = 
CeCe+1 if n is odd. 


(b) It is proved in [197] that 


[n/2I 
T,(123456) = 6 > (3) OT DIG TaN 


This is the largest k for which an exact formula is known for 
I,(12---k). 


This result is due to Herbert Wilf, who gave a generating function proof 
in [329]. Such an elegant result certainly asks for a direct bijective proof, 
but none is known to this day. 


This result is due to the present author [64]. The crucial idea of the 
proof is the following. Let B,, be the set of ordered pairs (p,q), where p 
is an involution on a subset s, of [2n], and q is an involution on the set 
[2n] — sp», the complement of s, in [2n]. 

Let B(n,k,r) be the subset of B,, consisting of pairs (p, g) so that neither 
p nor q has a decreasing subsequence longer than k. Note that here p is 
an involution of length r and q is an involution of length 2n — r. It is 
then clear that 


2 
|B(n,k,r)| = ( "nc edb iN), 
df 


the summand of the right-hand side of (7.5) indexed by r. Let B(n,k) = 
U, B(n, kyr). 

Let f be the involution defined on B,, that takes the largest fixed 7 point 
present in pUgq and if 72 is part of p, then moves it to q, and if 7 is part of 
q, then moves it to p. Let fn, be the restriction of f to the set B(n,k). 
It can then be shown that f,,, maps into B(n,k). So the only pairs 
(p,q) € B(n,k) whose contribution to the right-hand side of (7.5) is not 
canceled by the contribution of fn,~(p,q) are the pairs for which f(p, q) 
is not defined. It follows from the definition of fr, that these are the 
pairs in which both p and q are fixed point-free involutions. 


Apply the result of the previous Problem Plus with k = 3. It follows 
from the result of Problem Plus 2 that if p is a fixed-point free involu- 
tion, then the Standard Young Tableau corresponding to p has no odd 
columns. Therefore, ®2m41(1) = Zam(r). So #3(r) = xe(r), and so the 
result of the previous Problem Plus reduces to 


2n 2n 


3 @ 9(r)ava(2n —r) = S\(-1)" (*") 1,(1234) Jon (1234). 


r=0 r=0 
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Note that x2(r) is just the number of Standard Young Tableaux in which 
each column is of length two (since the columns are of even length not 
more than two). The number of such SYT is the Catalan number C;, 
(where r has to be even) as shown in Exercise 3. This shows that the 
two sides of (7.6) are equal to >; (”) (3”)CiC,_;. It is shown in Problem 


2% 
Plus 11, part (a), that this is precisely the formula for [,,(12345). 


This result is proved in [139] where the authors define an interesting 
notion of weight on fixed point-free involutions. 


It is proved in [332] that the number of these classes is (1 + o(1))n!/2. 
The result was also proved by El-Zahar and Sauer [163]. 


This result is proved in [232] in a more general context. A construction 
simply consist of all even permutations of length k + 1. Indeed, if two 
such permutations were not k-different, then they would differ only in 
a 2-cycle of the form (i k +1), but then they could not both be even. 
This lower bound is known to be optimal for k = 2 and k = 8, but is 
known to be not optimal for k = 4. 


This result was proved in [81]. Let n = 2m+ 1. Then it is shown in 
that paper that u,(321) > C?,. The proof uses the Robinson-Schensted 
correspondence, and a bijection of Claesson and Kitaev [121]. 


8 


Get Them All. Algorithms and 
Permutations. 


8.1 Generating Permutations 
8.1.1 Generating All n-Permutations 


If we want to write a computer program to test a conjecture concerning per- 
mutations, we need to have an efficient method to generate all n-permutations 
for our machine. If the conjecture only concerns permutations with some re- 
strictions, we can save a lot of time and effort by having a fast way to generate 
only those permutations with the required property. 

One can certainly list all permutations lexicographically. That is, let us 
define a partial order on the set of all n-permutations as follows. Let p = 
D1p2°**Pn < 4192°°*Qn if for the smallest index 7 for which p; 4 qi, we have 
pi <q. The total order defined on S$), is called the lexicographic order. So 
for instance, 34152 < 35412 since the smallest index for which p; and q; are 
different is 7 = 2, and po < q. 

It is obvious that the smallest element of $,, in the lexicographic order is the 
identity permutation 12---n. Therefore, in order to construct an algorithm 
to list all n-permutations in the lexicographical order, it suffices to have a 
method to find the permutation immediately following a given permutation 
p in the lexicographic order. Such a method is provided by the following 
proposition. Recall that in a poset we say that q covers p if p < q, and there 
isnorsothatp<r<4q. 


PROPOSITION 8.1 

Let p = pip2+++Pn be a permutation. Let i be the largest ascent of p. Then 
the permutation q covering p in the lexicographical order is given by q = 
Pip2*** Pi-1GiGit1°*' Qn, where q is the smallest element in {pi41, Pit2,+**Pn} 
that is larger than p;, and the string qi41°-:Gn contains the remaining entries 
of PiPi+1°**Pn im increasing order. 


Example 8.2 
If p = 2415763, then i = 4, and q = 2416357. 


DOI: 10.1201/9780429274107-8 347 


348 Combinatorics of Permutations, Third Edition 


PROOF Let q be as above, then p < q as p; < q. Let us assume 
that there exists an n-permutation r = rjr2---T, so that p< r <q. Then 
T1%2°++Ti-1 = pip2:+-pi—1 must hold, otherwise one of p< r andr < q could 
not be true. As we must have p; < rj; < q, and q; is the smallest remaining 
integer larger than p;, we must have either p; = r;, or r; = q;- Both are 
impossible, however. Indeed, in the first case, we could not have p < r, as the 
rest of p is decreasing, and in the second case, we could not have r < q, as 
the rest of q is increasing. 


Proposition 8.1 provides an obvious algorithm that generates all permuta- 
tions of length n, one-by-one, in the lexicographic order. The algorithm will 
stop when the index i, that is, the largest ascent of p, cannot be found; that 
is, when p = n(n — 1)---1, the maximum element of our total order. 


8.1.2 Generating Restricted Permutations 


A huge amount of research has been done on objects counted by the Cata- 
lan numbers. These objects, of which Richard Stanley lists over 150 different 
kinds in [297], have been enumerated according to various statistics, leading 
to interesting open problems. It is therefore desirable to be able to gener- 
ate these objects efficiently. We will show how to do this with 231-avoiding 
permutations. 

First, we define a total order H,, on the set of all 231-avoiding n-permutations 
as follows. Let p = LnR and q = L'nR’ be two n-permutations, where L de- 
notes the string on the left of n in p, and R denotes the string on the right of 
nin p, and L’ and R’ are defined analogously for g. Note that L and R could 
be empty. Now let |£| denote the length of L. We say that p <x, q if 


(a) |L| > |L’|, or 
(b) |L| =|L/|, and L < L/ in Hyz), or 
(c) |L| =|Z/|, and L =’, and R< R’ in Ajay. 


Example 8.3 

Let n = 8. Then p = 21348576 < 32184567 = q as L is of length four, while 
L’ is of length three, so rule (a) applies. Similarly, if r = 32148567, then p < r 
as 2134 < 3214, by the repeated application of rule (b) (213 < 321). Finally, 
if s = 21348765, then p < s by rule (c). 


Example 8.4 
For n = 4, the list of all elements of H, in increasing order is 1234, 2134, 
1324, 3124, 3214, 1243, 2143, 1423, 14382, 4123, 4213, 4132, 4312, 4321. 
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It is straightforward to see that H,, is indeed a total order as any two ele- 
ments of H,, are comparable. Futhermore, H, has minimum element 12---n 
and maximum element n---21. Note that the order H,, is quite left-heavy, 
that is, a little change made at the beginning of a permutation can influence 
the rank of the permutation much more than a little change at the end of the 
permutation. 

This observation suggests that we try to locate the unique element q cov- 
ering p = LnR in FH, by the following algorithm. 

Start with p= DnR. We will describe how to transform certain parts of p. 
Parts that are not mentioned in a given step are left unchanged by that step. 
We point out that 0-element and 1-element strings are considered monotone 
decreasing. 


(i) If R is not monotone decreasing, then replace p by R, redefine L and R 
for this new, shorter p, and go to (i) again. 


(ii) If R is monotone decreasing, but L is not, then replace p by L, redefine 
L and R for this new, shorter p, and go to (i) again. 


(iii) If L and R are both monotone decreasing, then move the first (leftmost) 
entry of L to the position immediately after the maximal entry of p. This 
creates new strings L and R; order the entries of L and the entries of R 
increasingly. The result of all these changes is the permutation q. 


The above algorithm will output a permutation for all inputs p 4 n(n—1)---1. 


Example 8.5 


e Let p= 21348576. Then L = 2134, and R = 576. So R is not monotone 
decreasing. Therefore step (i) applies. 


e Now we look at p = 576, and see that in it, both L and R are monotone 
decreasing. Therefore, step (iii) applies, and we obtain the string 756. 


e Concatenating this with the unchanged part of the original permutation, 
we get that g = 21348756. 


U 


Example 8.6 


e Let p = 215439876, then DL = 21543, and R = 876. So R is monotone 
decreasing, but L is not. So step (ii) applies. 


e Now p = 21548, so LD = 21 and R = 43. Both LZ and R are monotone 
decreasing, so step (iii) applies. 
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e In step (iii), we move 2 beyond 5, and rearrange L and R in increas- 
ing order. This gives the string 15234, which, concatenated with the 
unchanged part 9876, gives gq = 152349876. 


0 


THEOREM 8.7 
For any 281-avoiding n-permutation p other than the decreasing permutation, 
the algorithm described above outputs the unique n-permutation q that covers 
pin Hy. 


PROOF = Note that being 231-avoiding is equivalent to the property that 
all entries of Z are smaller than all entries of R, and that this holds recursively 
for L and R, and their recursively defined subwords. Steps (i) and (ii) of the 
algorithm will preserve this property as they operate within D and within R. 
Step (iii) moves the largest entry of L to the leftmost position of R, meaning 
that R now starts with its smallest entry. This again assures that the “entries 
of L are smaller than entries of R” property is preserved. Therefore q is indeed 
231-avoiding. 

Now we show that q indeed covers p. We are going to prove this by induction 
on n. The initial cases of n = 1 and n = 2 are obvious. Let us assume that 
we know that the statement is true for all positive integers less than n. 

If p = InR is such that at least one of L and R is not monotone decreasing, 
then it is straightforward to see by the induction hypothesis that q covers 
p. If £ and R are both monotone decreasing, then the only way to find 
permutations that are larger than p in H,, is by moving n closer to the front. 
If we want to find a permutation that covers p, then we must move n up 
by one position. As our permutations are 231-avoiding, this means that the 
entry that gets beyond n in this move must be the largest entry of L. As L is 
monotone decreasing, its largest entry is its leftmost one, just as prescribed 
in Step (iii). Finally, to get the smallest possible permutation with the new 
 aplaaa of n, we must order L and R increasingly. This completes the proof. 


Note that by the obvious bijections (reverse, complement), versions of the 
above algorithm will generate all 132-avoiding, 213-avoiding, or 312-avoiding 
permutations. It is less obvious how to generate all 321-avoiding or 123- 
avoiding permutations efficiently. 
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8.2 Stack-Sorting Permutations 


The task of sorting, that is, arranging n distinct elements in increasing or- 
der efficiently, is a central problem of computer science. There are various 
sorting algorithms, like merge sort, or heap sort, that can handle this task in 
O(nlogn) steps. It is also known that in the worst case, we indeed need this 
many steps. The interested reader is invited to consult a book on the Theory 
of Algorithms, such as [230], to learn these results. 

In this chapter we discuss some combinatorial sorting algorithms. In these 
algorithms, our goal is still to arrange n distinct objects (for the sake of 
brevity, the elements of [n]) in increasing order, but our hands will be tied by 
certain rules. 

Let p = pip2-::pn be a permutation, and let us assume that we want to 
rearrange the entries of n so that we get the identity permutation 123---n. 
This does not seem to be a mountainous task. However, this task will become 
more difficult if we are told that all we can use in our sorting efforts is a 
vertical stack that can hold entries in increasing order only (largest one at the 
bottom). We are even told how we have to use the stack (though this will 
rather help us than hold us back, as we will explain after Example 8.9). 

We note that there are alternative combinatorial sorting tools that lead 
to interesting problems; for instance, there are linear sorting devices that 
can receive or release entries on both ends. We will discuss some of these 
in Section 8.4. In the present section, we will devote all our attention to a 
certain greedy approach, which was first studied in detail by Julian West [325], 
though Tarjan did some work on the subject before that. Chronologically, this 
is not the earliest approach, but this is the one that has received the most 
attention. Since this approach seems to be the prevailing one, we will often 
refer to it simply as “stack-sorting,” instead of the longer, and more precise 
term “greedy stack-sorting.” We will mention some other approaches in later 
sections, and in the exercises. 

In order to stack-sort p = pip2--:Pn in the greedy way, we consider the 
entries of the input permutation p one-by-one. First take p,, and put it in the 
stack. Second, we take po. If po < p,, then it is allowed for p2 to go in the 
stack on top of pi, so we put p2 there. If pz > p,, however, then first we take 
p, out of the stack, and put it to the first position of the output permutation, 
and then we put po into the stack. We continue in this way: at step 7, we 
compare p; with the element r = pa,_, currently on the top of the stack. If 
pi <1, then p; goes on the top of the stack; if not, then r goes to the leftmost 
empty position of the output permutation, and p; gets compared to the new 
element that is currently on the top of the stack. The algorithm ends when 
all n entries passed through the stack and are in the output permutation s(p). 

We are now ready to announce the most important definition of this section. 
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output stack input 
3142 
3 142 
1 42 
1 3 42 
13 4 2 
13 2 
4 
132 4 
1324 


FIGURE 8.1 
Stack-sorting f = 3142. 


DEFINITION 8.8 _ If the output permutation s(p) defined by the above 
algorithm is the identity permutation 123---n, then we say that p is stack- 
sortable. 


Example 8.9 
Figure 8.1 shows the stages of stack-sorting the permutation 3142. We con- 
clude that s(p) = 1324, therefore 3142 is not stack-sortable. 0 


If you think that our sorting algorithm is too arbitrary, in that it requires 
the entries in the stack to be in increasing order, or that it tells us when 
an entry should enter or leave the stack, consider the following. If at any 
point of time, the entries in the stack were not in increasing order, then we 
could surely not obtain the identity permutation at the end since no entry x 
currently below an entry y in the stack could pass y in the output. So the 
monotone increasing requirement for the stack is not a real restriction. Now 
that we know this, let us look at the other requirement. That requirement 
says that if pj <r, where r is currently on the top of the stack and p; is the 
next entry of the input, then p; goes on the top of the stack, and if not, then r 
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goes to the output. We claim that this is not a real restriction either. Indeed, 
if p; is the larger one, then it cannot go on top of r as that would destroy the 
increasing property of the stack. If p; is the smaller one, then r should not go 
to the output before p; does as that would create an inversion in the output. 
This shows that if a permutation is sortable by a stack, our algorithm will 
output the identity permutation, and the mentioned criteria are not limiting 
our possibilities. 

What permutations are stack-sortable? This is one of the very few easy 
questions of this area. 


PROPOSITION 8.10 
A permutation is stack-sortable if and only if it is 231-avoiding. 


PROOF Let p be stack-sortable, and let us assume that entries b, c, 
and a of p form a 231-pattern in this order. Then b would enter the stack at 
some point, but would have to leave it before the larger entry c could enter it. 
As a could only enter the stack after c, it is clear that a would arrive to the 
output after b did, implying that s(p) would contain the inversion ba, which 
is a contradiction. 

Conversely, let us assume that p is not stack-sortable. Then the image s(p) 
contains an inversion yx. That means y > x, so y must have entered the stack 
before x did, and so y must have left the stack before x even arrived there. 
What forced y to leave the stack before x arrived? By our algorithm, this 
could only be an entry z that is larger than y and is located between y and x 
in p. However, in that case yzx was a 231-pattern in p. 


Note that we have in fact proved the following statement that will be useful 
in later applications. 


COROLLARY 8.11 

The entries u < v of p will appear in s(p) in decreasing order if and only 
if there is a 231-pattern in p whose leftmost entry is v and whose rightmost 
entry 1s U. 


8.2.1 2-Stack-Sortable Permutations 


We have seen that only C,, < 4” permutations of length n are stack-sortable 
out of n! total permutations, meaning that greedy stack sorting in itself is not 
very efficient. However, we can repeat the entire sorting algorithm by sending 
our output s(p) through the stack again. 


DEFINITION’ 8.12 The permutation p is called 2-stack-sortable if 
s(s(p)) = s?(p) = 123--+n. Similarly, p is called t-stack-sortable if s‘(p) = 
123---n. 
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It is not hard to see that no n-permutation will require more than n — 1 
applications of s to be sorted. In other words, all n-permutations are (n — 1)- 
sortable. See Exercise 5 for a proof of this fact. 

Encouraged by the simple result of Proposition 8.10, one might try to char- 
acterize 2-stack-sortable permutations by classic pattern avoidance. These 
efforts, however, are bound to fail. Indeed, sets of permutations that consist 
of all permutations avoiding a given set of patterns are always permutation 
classes. That is, if p is in such a set, then so are all the substrings of p. 
However, this is not true for 2-stack-sortable permutations. 


Example 8.13 

The permutation g = 35241 is 2-stack-sortable. Indeed, s*(p) = s(32145) = 
12345. On the other hand, its substring p = 3241 is not 2-stack-sortable, as 
s*(p) = s(2314) = 2134. 


For 2-stack-sortable permutations, some characterization is nevertheless 
possible. This is the content of the next lemma. 


LEMMA 8.14 

[825] A permutation is 2-stack-sortable if and only if it does not contain a 
2341-pattern, and it does not contain a 3241-pattern, except possibly as part 
of a 8524 1-pattern. 


PROOF Let us assume first that p is 2-stack-sortable. Then by Propo- 
sition 8.10, its image s(p) must be 231-avoiding. However, if bcda were a 
2341-pattern in p, then bca would be a 231-pattern in s(p). Similarly, if cbda 
were a 3241-pattern in p that is not part of a 35241-pattern, then s(p) would 
contain a 231-pattern. This pattern could be bca (if there is no 231-pattern 
starting with c and ending with b) or cfa (if f is the largest entry between c 
and 0 in p so that cfb is a 231-pattern). 

Let us assume now that p is not 2-stack-sortable. Then s(p) is not stack- 
sortable; therefore, it must contain a 231-pattern. Let bca be such a pattern. 
Then it follows from Corollary 8.11 that there is a 231-pattern bda and a 
231-pattern cea in p. If b is on the left of c in p, then we see that bcea is 
a 2341-pattern in p. If b is on the right of c in p, then again by Corollary 
8.11, there cannot be any entry between c and b in p that is larger than c. 
However, that implies that cbea is a 3241-pattern in p that is not part of a 
35241-pattern, completing our proof. 


The problem of finding an exact formula for the number W2(n) of 2-stack- 
sortable n-permutations is a difficult and fascinating one. 
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THEOREM 8.15 
For all positive integers n, the identity 


2(3n)! 


Wan) (n+ 1)\(2n 4+ 1)! 


holds. 


The above formula was first conjectured in a Ph.D. thesis by Julian West 
[325], who pointed out that these numbers also enumerate nonseparable rooted 
planar maps on n + 1 edges. The first proof was provided five years later 
by Doron Zeilberger [338], who used a computer to find the solution to a 
degree-9 functional equation. Two other proofs [149], [204] have been found 
later. Both construct fairly complicated bijections between the set of 2-stack- 
sortable n-permutations and the aforementioned planar maps. The latter 
have been enumerated by Tutte [313] in 1963. Finally, a bijection between 2- 
stack-sortable permutations, and a certain class of labeled trees, the so-called 
G(1, 0)-trees, was discovered [127], but a simple proof is yet to be found. Even 
a simple proof for the much weaker claim that W2(n) < (°") could be very 
useful as it could provide useful insights towards finding upper bounds for the 
numbers of t-stack-sortable permutations, for t > 2. 

We close our introduction to 2-stack-sortable permutations with a fascinat- 
ing fact that has not been satisfactorily explained yet. It is a classic result of 
Kreweras that does not look related at first sight. 


THEOREM 8.16 

The number of lattice paths starting at (0,0), ending at (7,0), and using 3n+2% 
steps, each of which is equal to either (1,1), or (0,—1), or (—1,0), that never 
leave the first quadrant is 


4” (27 + 1) 21\ (3n + 2% 
(n+i+1)(Qn+2i+1)\a n ; 
This result has several proofs. The most direct one is by Olivier Bernardi 
[36]. See [88] for a history of the problem and for other proofs. If we set 
i = 0, then we get that the number of such lattice paths that end at (0,0) in 


3n steps is precisely 27"~!. W(n). A direct proof of this fact could lead to a 
simple proof for the formula for the numbers W2(n). 


8.2.2 t-Stack-Sortable Permutations 


Let W;(n) be the number of t-stack-sortable n-permutations, that is, permu- 
tations p for which s‘(p) = 12---n. It is very difficult to enumerate these 
permutations if t > 2. See Exercise 9 for an upper bound that holds for all t, 
but is far from what seems to be the truth. 
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2n 3n 
Encouraged by the exact formulae W;(t) = (2) and W2(t) = one 


we might think that in the general case, we could hope for a formula like 


co) 


Wi(n) = ; (8.1) 


p(n) 


where p(n) is a polynomial with rational coefficients. Numerical evidence, 
however, does not support this conjecture. Indeed, if (8.1) held for t = 3 
and some polynomial p(n), then the number W3(n) could have no prime 
factor larger than 4n. This contradicts the available data since W3(6) = 606, 
which has prime factor 101. The first few values of W3(n) can be found in 
the Encyclopedia of Integer Sequences as sequence A134664. The fact that 
W3(6) = 606 follows from Exercise 8. 

Next, one could ask if at least the inequality W:(n) < (ere) holds. Re- 
cently, Colin Defant, Andrew Elvey Price, and Anthony J. Guttmann [138] 
proved that the answer to that question is negative. They showed that the 
exponential growth rate of the sequence W3(n) is at least 9.4854, which 
is larger than the exponential growth rate of the sequence ea? which is 
256/27 = 9.4815. 

The following simple observation narrows the field of permutations that 
have a chance to be t-stack-sortable. 


PROPOSITION 8.17 
If the permutation p is t-stack-sortable, then p avoids the pattern 23---(t+ 
2)1. 


PROOF _ If pcontains such a pattern, then the entries forming a copy of 
such a pattern will form a 23---¢1(¢+1)-pattern in s(p), a 23---(¢—1)1¢(¢+1)- 
pattern in s?(p), and so on. Finally, they will form a 23145 --- (t+ 1)-pattern 
in s*-'(p), implying that s‘~'(p) contains a 231-pattern, and is not stack- 
sortable. So its image, s*(p), is not the identity permutation. 


8.2.2.1 Equivalent Algorithm 


Consider the following modification of the t-stack sorting operation. Instead 
of passing a permutation through a stack ¢ times, we pass it through t stacks 
placed next to each other in series as follows. The first stack operates as the 
usual stack except that when an entry x leaves it, it does not go to the output 
right away. It goes to the next stack if « < 7, where 7 is the entry on the top 
of the next stack, or if the next stack is empty. If 7 < x, then x cannot move 
until 7 does. 

The general step of this algorithm is as follows. Let 5 ,5S2,--- ,S; be the 
t stacks, with a; being the entry on top of stack $;. If the next entry x of 
the input is smaller than a,, we put x on top of S;. Otherwise, we find the 
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smallest 7 so that a; can move to the next stack (that is, that a; < aj41 or 
Si+1 is empty), and move a; on top of 5,41. If we do not find such 3, or if 
S1,52,°-- , 5-1 and the input have all been emptied out, then we put the 
entry on the top of S; into the output. 

We will call this algorithm the right-greedy algorithm, as it always makes 
the rightmost move possible as the entries move from the input on the right 
to the output on the left. 

We can describe the movement of the entries of the input permutation p 
through the stack by stack words. If t = 1, then there are just two kinds of 
moves, an entry either moves in the stack or out of the stack. Let us denote 
these steps with letters A and B, respectively. Then the movement of all 
entries of p is described by a stack word consisting of n copies of A and n 
copies of B in which for all 2, the ith A precedes the ith B. The number of 
such words is well-known to be C, = (*”)/(n +1). On the other hand, if p 
is stack-sortable, then its output is the identity, so given the stack word of p, 
we can uniquely recover p. It is easy to prove by strong induction that each 
word that satisfies the conditions described in this paragraph is indeed the 
stack word of a stack-sortable permutation, so this is a stack word proof of 
the fact that Wy(n) = Ch. 

In general, if there are ¢t stacks, then there are t+ 1 different kinds of moves. 
Therefore, the movement of p through the t + 1 stacks can be described by 
a word of length (tf + 1)n that consists of n copies of each of t + 1 different 
letters. 

In what follows, we will consider the case of t = 3, when there are four 
letters, to prove a significantly sharper upper bound for the exponential order 
of the sequence W3(n). Let A denote the move of an entry from the input to 
the first stack, let B denote the move of an entry from the first stack to the 
second stack, let C' denote the move of an entry from the second stack to the 
third stack, and let D denote the move of an entry from the third stack to 
the output. 

Note that we will also call stack words 3-stack words or 2-stack words when 
we want to emphasize the number of stacks that are used to sort a given word. 
Also note, for future reference, that for all t, the descents of p are in bijective 
correspondence with the AA factors of p. (An XY factor of a word is just a 
letter X immediately followed by a letter Y.) 

We will identify 3-stack-sortable permutations with their stack words. We 
can do that since if p is 3-stack-sortable, then its image under the 3-stack 
sorting algorithm is the identity permutation, so given the stack word of p, 
we can uniquely recover p. 


PROPOSITION 8.18 
Let w be a 3-stack word of a permutation. Then all of the following hold. 


1. There is no BB factor in w. 
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2. There is no CC factor in w. 
8. There is no BAB factor in w. 
4. There is no CBAIC factor in w, where j > 0. 


PROOF Each of these statements holds because otherwise the entries in 
the second or third stack would not be increasing from the top of the stack 
to the bottom of the stack. 


PROPOSITION 8.19 
Let w be a 3-stack word of a permutation. Then all of the following hold. 


1. There is no DA factor in w. 
2. There is no DB factor in w. 
8. There is no CA factor in w. 


PROOF Each of these statements holds because of the greediness of 
our algorithm. For instance, a D cannot be followed by an A, since the 
move corresponding to D did not change the content of the first stack, so 
if the A move was possible after the D move, it was possible before the D 
move, and therefore, it would have been made before the D move. Analogous 
considerations imply the other two statements. 


Note that the conditions given in Propositions 8.18 and 8.19 are necessary, 
that is, they must hold in 3-stack words of all permutations, but they are 
not sufficient. In other words, if a word satisfies all these conditions, it is not 
necessarily the 3-stack word of a permutation. 

Let w be a 3-stack word of a 3-stack-sortable permutation p, and let v = 
v(w) be the subword of w that consists of the letters B, C and D in w. In 
other words, v = v(w) is the word obtained from w by removing all copies of 
the letter A. This can create BB factors in v, even though there were no BB 
factors in w. 

Note that v describes how the stack sorted image s(p) of p traverses the 
second and third stacks. Note that as p is 3-stack-sortable, s(p) is 2-stack- 
sortable. So v is the 2-stack word of the 2-stack-sortable permutation s(p) 
over the alphabet {B,C,D}. Therefore, there are W2(n) = aot (*") 
possible choices for v. 

Furthermore, every descent of s(p) bijectively corresponds to a BB-factor 
of v. The number of 2-stack-sortable permutations of length n with k — 1 
descents is known (see Problem Plus 8.1) to be 


(n+k—1)!(2n —k)! 


Wan, k—1) = 74 ek Dan Dk DT 


(8.2) 
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LEMMA 8.20 
The number W3(n) of 8-stack-sortable permutations of length n satisfies the 
inequality 


(n+1)/2 


(n+k—1)!(2n —k)! 2n — 2k 
Wa(n) s 2, k(n +1—b)l(Qk—1)Qn—2k+ 1)! ( oo i, 


PROOF _ Let us count all such permutations with respect to the number 
of descents of their stack sorted image s(p). If s(p) has k—1 descents, then its 
2-stack word v has k — 1 factors BB. In order to recover the 3-stack word w 
of p, we must insert n copies of A into v so that we get a valid 3-stack word. 
As BB factors and BAB factors are not allowed in w, we must insert two 
copies of A into the middle of every BB factor, and we also have to put one 
A in front of the first B. After that, we still have n—2(k-—1)-1l=n-2k+1 
copies of A left. We can insert these in n possible slots only, namely on the 
left of the first B, and immediately following any B except the last one. (This 
is because Proposition 8.19 tells us that there are no CA or DA factors in 
w.) Therefore, by a classic balls-and-boxes argument, the number of ways to 
place all copies of A is at most 


n—-2k+1+n—-1\ — (2n—2k 
n—1 ~\n-1 J 


As there are W2(n,k — 1) choices for v, the proof is complete by summing 


over all possible values of k. | 


It is easy to prove that limn+.. ~/W3(n) exists. Now we are ready to prove 
an upper bound for that limit. 


THEOREM 8.21 
The inequality 


lim %/W3(n) < 12.53296 


n—-Co 


holds. 


PROOF As Lemma 8.20 provides an upper bound for W3(n) as a sum of 
less than n summands, it suffices to prove that the largest of those summands 
is of exponential order 12.539547. In order to do that, we use Stirling’s formula 
that states that m! ~ (m/e)™V2mm, so limm +. Vm! = m/e. Let w3(n, k) 
denote the number of 3-stack-sortable permutations p of length n so that 
s(p) has k descents. Setting k = na, with x € (0,1), and applying Stirling’s 
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formula to each factor of the bound in Lemma 8.20, this leads to the equality 


g(z):= Jim, w3(n, zn) 


2®-(1—z)1-2. (22)? . (2 — 2z)2-22 . (1 — 2z)1-2 


=(1+2):(2—z)?-*+2-*-@—2)-*- (1-22). (+) 


The function g takes its maximum when g/(z) = 0, which occurs when 


1 1 
— & 0.2883918927. 


1 
a ee eS er 2 
12 ( ) 4-(274+12-V417)1/3 4 


For that value of z, we get g(z) = 12.53296, completing the proof. | 


Colin Defant [132] proved Theorem 8.21 using a different method. His 
method was more complicated, but he was able to use it prove that the in- 


equality 
lim */Wa(n) < 21.97225 
noo 


holds as well. 


8.2.2.2 Symmetry 


Let W;(n,k) be the number of t-stack-sortable permutations with & descents. 
We propose to fix n and t, and investigate the sequence W;(n, k)o<pk<n—1- For 
instance, if n = 5, and t = 3, then we find the sequence 1, 25, 62, 25,1, while 
for n = 4 and t = 1, we get the sequence 1,6,6,1. A look at further numer- 
ical evidence suggests several interesting properties of these sequences. The 
simplest one is symmetry, that is, it seems that W;(n,k) = Wi(n,n—1-—k), 
or in other words, there seem to be as many t-stack-sortable n-permutations 
with & descents as with k ascents. In fact, for t = n — 1, the statement is 
obvious, considering Exercise 5, while for t = 1 and ¢ = 2 we can verify that 
the statement is true using the known explicit formulae. See Problem Plus 
1 of Chapter 4 for the relatively simple formula for t = 1, and see Problem 
Plus 1 of this chapter for the more complex formula for t = 2. These are not 
sporadic special cases. 


THEOREM 8.22 
For all fixed n and t, we have W;(n, k) = Wi(n,n —1—k). 


PROOF There seems to be no trivial reason for this symmetry. Indeed, 
the usual symmetries of permutation classes (reverse, complement) that turn 
ascents into descents do not preserve the t-stack-sortable property, even when 


Get Them All. Algorithms and Permutations. 361 


7 


FIGURE 8.2 
The decreasing binary tree of p = 3762514. 


t = 1. In order to prove our theorem, we need to find a more subtle symmetry 
that turns ascents into descents, and preserves the t-stack-sortable property 
for any t. 

We will use the notion of decreasing binary trees, that we have discussed in 
Section 6.3. For easy reference, here is the definition again. 


DEFINITION 8.23 Let p = InR be an n-permutation, where L and 
R denote the (possibly empty) substrings of p on the left and right of n. The 
decreasing binary tree T(p) of p is the binary plane tree with root n, left subtree 
T(L), and right subtree T(R). 


Example 8.24 
The decreasing binary tree of the permutation 3762514 is shown in Figure 8.2. 
Here we have s(p) = 3214567. 


It is easy to prove (Exercise 31 of Chapter 1) that decreasing binary trees 
on n nodes are in bijection with n-permutations. 

There are two reasons for which decreasing binary trees are fitting to the 
task at hand. First, just as their remote cousins, the binary plane trees, they 
encode the number of descents. Indeed, the reader is invited to prove the 
simple fact that the number of descents of p is equal to the number of right 
edges (that is, edges that go from northwest to southeast) of T(p). Second, 
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the stack sorted image s(p) can easily be read of T(p), thanks to the following 
proposition. 


PROPOSITION 8.25 
Let p = InR be an n-permutation. Then we have 


s(p) = s(L)s(R)n. (8.3) 


PROOF _ By the definition of the stack sorting algorithm, the entry n can 
only enter the stack when the stack is empty, that is, when all entries on the 
left of m passed through the stack. This shows that s(p) will start with the 
string s(Z). Once L passed through the stack, n will enter. However, as n is 
larger than any other entry, n will not be forced out from the stack by any 
other entry. So n will stay at the bottom of the stack until all other entries 
have passed through it. This shows why s(p) will continue with the string 
s(R), and then end in the entry n. 


This important “left-right-n” property could be used to define the stack 
sorting operation. Here is what it means in the context of decreasing binary 
trees. 


COROLLARY 8.26 

Given T(p), we can read off s(p) from T(p) by reading the nodes of T(p) in 
postorder. That is, first we read the left subtree of the root, then the right 
subtree of the root, and then the root itself. Each subtree is read by this same 
rule. 


PROOF _ Immediate by induction on n, using Proposition 8.25. See Figure 
8.2 for an example. 


Now we are ready to define the bijection that will prove Theorem 8.22. We 
will define this bijection f in terms of decreasing binary trees as this approach 
shows the “real reason” for the nice symmetries we are going to prove. As 
there is a one-to-one correspondence between these trees and n-permutations, 
f can easily be interpreted in terms of permutations as well. 


DEFINITION 8.27 Let T(n,k) be the set of decreasing binary trees on 
n vertices that have k right edges, and let T(p) € T(n,k). For each vertex v 
of T(p) do as follows. 


(a) If v has zero or two children, leave the subtrees of v unchanged. 
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FIGURE 8.3 
The tree f(T(p)) for p = 3762514. 


(b) Ifv has a left subtree only, then turn that subtree into a right subtree. 
(c) Ifv has a right subtree only, then turn that subtree into a left subtree. 


Let f(T(p)) be the tree we obtain from T(p) in this way. 


Example 8.28 
Let p = 3762514, then T(p) is the tree shown in Figure 8.2. 

To construct f(T (p)), note that the nodes corresponding to 3, 7, 2, 5, and 
1 have either zero or two children, so they are left unchanged. The node 6 has 
a right subtree only, so that subtree is turned into a left subtree, while the 
node 4 has a left subtree only; therefore, that subtree is turned into a right 
subtree. We end up with the tree f(T (p)) shown in Figure 8.3. 

We then read off that the corresponding permutation is f(p) = 3725416. 


It is then clear that f(T(p)) € T(n,n-—1—k), so f has the desired effect 
on the number of descents of the permutation p. 

You could say, “Fine. So you have proved in an ever so sophisticated way 
that there are as many n permutations with k descents as there are with k 
ascents. I can do it by just reversing all permutations.” At this point, you 
would be right in saying so. However, our bijection f preserves the t-stack- 
sortable property for any t, implying that the restriction of f to the set of 
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t-stack-sortable n-permutations is just what we need to prove our theorem. 
This is the content of the next lemma. 

For a permutation p, we will write f(p) for the permutation whose decreas- 
ing binary tree is f(T (p)). In other words, f(T'(p)) = T(f(p)). 


LEMMA 8.29 
For any fixed n and t, the n-permutation p is t-stack sortable if and only if 
f(p) ts t-stack-sortable. 


PROOF We claim that 


s(p) = s(f(p)), (8.4) 


that is, s(p) is (¢ — 1)-stack-sortable if and only if s(f(p)) is, which clearly 
implies the statement of the Lemma. 

Intuitively speaking, what (8.4) says is that pushing some lonely left edges 
to the right or vice versa does not change the postorder reading of T(p). Let 
us make this argument more precise. 

We prove (8.4) by induction on n. For n = 1 and n = 2, the formula 
obviously holds. Now let us assume that we know the statement for all non- 
negative integers less than n. 

If the root of T(p) has two children, then p = LnR, and the postorder 
reading of T(p) is just the concatenation of the postorder reading of T(L), the 
postorder reading of T(R), and n. By our induction hypothesis, the postorder 
reading of T(L) is the same as that of T(f(Z)), and the postorder reading of 
T(R) is the same as that of T(f(R)). Therefore, as the root of T'(f(p)) has two 
children, and they are roots of the trees T(f(Z)) and T(f(R)), the postorder 
reading of T(p) and that of T(f(p)) are identical as they are concatenations 
of identical strings. 

If the root of T(p) has a left child only, then p = Ln, and the postorder 
reading of T(p) is that of T(L), with n added to the end. In this case, the 
root of T(f(p)) has only one child, and that is a right child. This child is the 
root of a subtree isomorphic to T(L). This is no problem, however. As the 
root has only one child, the postorder reading of T(f(p)) is just the postorder 
reading of this one subtree, that is, T(L), with n added to end. This proves 
our claim. If the root of T(p) has a right child only, the argument is the same 
with “left” and “right” interchanged. 


Therefore, the restriction of f to the set of t-stack-sortable n-permutations 
maps to the set of t-stack-sortable n-permutations. We have seen that f maps 
permutations with k descents to permutations with & ascents, so our theorem 
is proved. 
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8.2.3. Unimodality 


A useful feature of the proof of Theorem 8.22 was that we could prove a 
relatively strong statement about t-stack-sortable permutations, without even 
knowing what they are, in the sense of an explicit characterization. In this 
subsection, we are continuing this line of work. 


THEOREM 8.30 
For all fired n and t, the sequence W;(n, k)o<k<n—1 is unimodal. 


Before we prove the theorem, we point out that while the special cases t = 1 
and t = 2 can again be verified by the aforementioned explicit formulae, the 
special case t = n — 1 is not nearly as obvious as it was when we proved 
symmetry. 


PROOF _ As we saw in Theorem 8.22, the sequence at hand is symmetric, 
and therefore, it suffices to prove that W:(n,k) < Wi(n,k +1) for k < [(n- 
3)/2]. 

We resort to decreasing binary trees again, and the outline of our proof will 
also be somewhat similar to that of Theorem 8.22. We will first define an 
injection z: T(n,k) ~ T(n,k +1). Then we will show that z preserves the 
t-stack-sortable property, completing our proof. 

Let T be any decreasing binary tree on n nodes. We define a total order of 
the nodes of T as follows. Let us say that a node v of T is on level j of T if the 
distance of v from the root of T is 7. Then our total order consists of listing 
the nodes on the highest level of T’ going from left to right, then the nodes on 
the second highest level left to right, and so on, ending with the root of T. 

Let T; be the subgraph of T induced by the smallest 7 vertices in this total 
order. Then T; is either a tree or a forest with at least two components. If T; 
is a tree, then f(T;) is a tree as described in Definition 8.27. If T; is a forest, 
then we define f(T) as the plane forest whose hth component is the image of 
the hth component of T;. 

Now let & < |(n — 3)/2], and let T € T(n,k). We define z(T) as follows. 
Take the sequence T),T2,--- ,T,. Denote by ¢(T’) the number of left edges of 
the forest T, and by r(T) the number of right edges of the forest T. Find the 
smallest index 7 so that ¢(T;) — r(T;) = 1. We will now explain why such an 
index always exists. If T> is a left edge, then ¢(T2) — r(Tz2) = 1—0 = 1, and 
we are done. Otherwise, at the beginning we have ¢(T2) — r(T2) < 1, while at 
the end we have 


&(In) — r(In) > ((n— 1) — [(n — 8)/2]) — [(n — 8)/2] > 1, 


because of the restriction on k. So at the beginning, ¢(Z}) —r(T;) is too small, 
while at the end, it is too large. On the other hand, it is obvious that as 7 
changes from 2 to n, at no step could ¢(T;) — r(Z;) “skip” a value as it could 
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FIGURE 8.4 
A decreasing binary tree and its image under z. 


only change by 1. Therefore, for continuity reasons, it has to be equal to 1 at 
some point, and we set 7 to be the smallest index for which this happens. 

Now apply f to the forest T;, and leave the rest of T unchanged. Let z(T) 
be the obtained tree. Before we prove that z is an injection, let us consider 
an example. 


Example 8.31 

Let T be the decreasing binary tree shown on the left of Figure 8.4. The small 

numbers denote the ranks of the vertices in the total order defined above, and 

the large numbers are the entries of the permutation associated with p. 
Then the smallest index 7 for which ¢(T;) — r(T;) = 1 is 1 = 2, so we have to 

apply f to the tree 75, the tree that is encapsulated in Figure 8.4. The image 

z(T) is then shown on the right-hand side of Figure 8.4. 


Let us return to the proof of the injectivity of z. 


LEMMA 8.32 
For all k < |(n —3)/2]|, the function z described above is an injection from 
T(n,k) into T(n,k+1). 


PROOF It is clear that z maps into T(n,k +1) as z consists of the 
application of f to a subgraph T; in which left edges outnumber right edges 
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by one. We know that f turns left edges into right edges and vice versa, so z 
indeed increases the number of right edges by 1. 

To see that z is an injection, note that z(T;) is the smallest subforest of 
z(T) that consists of the first few nodes of z(T) in the total order of all nodes 
and in which right edges outnumber left edges by one. 

Now let U € T(n,k+1). If U does not have a subforest that consists of the 
first few nodes of U in the total order of all nodes and in which the right edges 
outnumber the left edges by one, then by the previous paragraph, U has no 
preimage under z. Otherwise, the unique preimage z~'(U) can be found by 
finding the smallest such subforest, applying f to it, and leaving the rest of 
U unchanged. 


So the injection z provides yet another proof of the fact that the Eulerian 
numbers form a unimodal sequence. In order to show the relevance of z to 
t-stack-sortable permutations, we still have to show that z preserves the t- 
stack-sortable property. This will be done again in a way that is quite similar 
to the proof of Theorem 8.22. Denote by s(T7’) the postorder reading of the 
decreasing binary tree T. 


LEMMA 8.33 
For any decreasing binary trees T for which z(T) is defined, we have 


s(T) = s(z(T)). 


PROOF _ The only place T and z(T) differ is in their first 7 vertices in 
the total order. However, even those parts agree in their postorder readings 
as they are images of each other under the postorder-preserving bijection /. 


The proof of Theorem 8.30 is now immediate. Restrict z to the set of de- 
creasing binary trees corresponding to t-stack-sortable n-permutations with k 
descents, where k < |(n — 3)/2|. By the two preceding lemmas, this restric- 
tion maps injectively into the set of trees corresponding to t-stack-sortable 
n-permutations with k + 1 descents, proving our claim. 


Petter Brandén [94] has provided an alternative proof for Theorem 8.30 as 
part of a more general framework. 


8.2.3.1 Log-Concavity 


Having seen that the sequence W;(n, k)o<k<n—1 is unimodal for any fixed n 
and k, it is natural to ask whether these sequences are log-concave as well. 
Surprisingly, a general answer for this question is not known. It is rather 
rare in enumerative combinatorics to have a sequence that arises naturally 
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from some enumeration problem, is known to be symmetric and unimodal, 
and is not known (but widely thought) to be log-concave. So our sequences 
Wi(n, k)o<k<n—1 are examples of a rare phenomenon. 

There are some sporadic results, though. For ¢ = 1 and t = 2, the result 
is immediate from the explicit formulae for our numbers. For t = n — 1, the 
statement is equivalent to the log-concavity of the Eulerian numbers. See 
Exercise 13 for the special case of t = n — 2. 


8.2.3.2 Real Zeros 


Numerical evidence seems to suggest the even stronger statement that for any 
fixed n and t, the polynomial 


n-1 
Wnt(@) = S- W,(n, k)ak*4 (8.5) 
k=0 


has real zeroes only. This observation has not been proved to hold in all cases 
yet. The only simple special case is when t = n — 1, because in that case the 
statement simply claims that the Eulerian polynomials have real zeroes only. 

The special cases of t = 1 and t = 2 are not obvious, even though we have 
exact formulae for the numbers W;(n, k). In both of these special cases, Petter 
Brandén proved that W,,4(a) has real roots only [93]. In that same article, 
Brandén developed the analytic tools to prove that Wy n—2(#) has real roots 
only, marking the fourth special case for which the real-roots-only conjecture 
is known to be true. 


Dr 


8.3. Pop-stack sorting 


Pop-stack sorting is the following version of stack-sorting. The entries of the 
permutation to be sorted, p, enter a specialized stack, called a pop-stack one- 
by-one, just as in stack-sorting, as long as the entries in the pop-stack form 
an increasing sequence from top to bottom. However, when this is no longer 
possible, that is, when the next entry of p is larger than the entry on the top of 
the pop-stack, then the entire pop-stack must go into the output. We call the 
permutation p pop-stack-sortable if the image pop(p) of p by this deterministic 
algorithm is the identity permutation. 


Example 8.34 
The permutation p = 3215476 is pop-stack-sortable as the pop-stack will 
empty out after the third, fifth, and seventh entry enters it. 
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Example 8.35 
The permutation p = 4213756 is not pop-stack-sortable, since pop(p) starts 
with the subsequence 124. 


It is not difficult to characterize pop-stack-sortable permutations in terms 
of pattern avoidance. 


THEOREM 8.36 
A permutation p is pop-stack-sortable if and only if p avoids both 281 and 812. 


PROOF Let p = r17T2---rTz be the unique decomposition of p into 
non-extendible descending runs (decreasing sequences of consecutive entries). 
Then pop(p) = a a2ax%, where a; is the reverse of r;. Therefore, pop(p) = 
12---n if and only if all entries in a; are smaller than all entries of a;, when- 
ever i < j. That happens when p is a layered permutation. (See Definition 
4.94.) The solution of Exercise 13 of Chapter 5 shows that a permutation is 
layered if and only if it avoids both 231 and 312. fl 


If p is not the identity permutation, then pop(p) has less inversions than 
p. This implies that applying the pop-stack sorting operation to p a sufficient 
number of times will result in the identity permutation, no matter what p is. 
But how many times do we have to send p through the pop-stack to sort it? 
In other words, what is the smallest integer t so that pop'(p) = 12---n, for 
all n-permutations p? As before, pop’(p) is defined recursively, by the rule 
pop'(p) = pop(pop’ 1 (p)), and we say that p is t-pop-stack-sortable if pop’ (p) 
is the identity permutation. 

Recall that for the regular stack sorting operation s, the answer to the 
analogous question is n — 1. So it might seem plausible to think that for the 
pop-stack sorting operation, the answer will be a higher number, because pop 
sorts only 2”~! permutations of length n in one step, while s sorts almost 4” 
permutations of length n in one step. 

The following theorem shows that this intuition is false. 


THEOREM 8.37 
Every permutation of length n is (n — 1)-pop-stack-sortable. 


In other words, in the worst case scenario, pop-stack sorting is just as 
efficient as stack sorting! Somewhat surprisingly, the theorem is a direct 
consequence of a result of the combinatorial geometer Peter Ungar [314] from 
1982. In what follows, we present a version of that proof that was given by 
Michael Albert and Vincent Vatter [10]. 

The main idea of the proof is the following. We will show that after n — 1 
applications of pop, the k smallest entries of p will precede the n — k largest 
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entries of p, for all k. This clearly implies that pop"!(p) is the identity 
permutation. In order to do this, we will sometimes disregard all properties 
of the entries of p other than being larger than k, by replacing them by 0s 
and ls. We will also define a nondeterministic sorting operation on binary 
words called tumble so that pop(p) will be just one of the possible outputs of 
the tumble operation on p. Finally, we will define a deterministic operation 
on binary words called flip which will lead to the worst possible output of 
tumble, and show that even flip is strong enough to sort all binary words of 
length n in n — 1 iterations. 

In order to proceed in that direction, we introduce the following notation. 
For a permutation p = p1p2--+ Dn, set 


Oifp<k, 


P |e (i) = 
lifp>k. 


Example 8.38 
Let p = 356142, and let k =3, then p|3=011010. [ 


In other words, p |, is a binary word that encodes which positions of p 
contain entries larger than k. Let us call p |, the k-truncation of p. 

Let w be a binary word, that is, a word over the alphabet {0,1}. A factor 
in a word is just a substring of consecutive entries. We define tumble, a 
nondeterministic operator on the set of all finite binary words as follows. The 
tumble of w is the set T(w) of all words that can be obtained from w by the 
following steps. 


1. Choose a set of non-overlapping 1'0/ factors with i and j being positive 
so that the chosen factors together cover all 10-factors of w. 


2. Reverse all those chosen 1°0/ factors in place. 


Example 8.39 

Let w = wyw2---wg = 11100101. Reversing the strings waw3w4 and wew7, 
we get the word 10110011 € T(w). Reversing the strings wi w2w3wW4ws we get 
the word 00111011 € T(w). The reader is invited to verify that T(w) consists 
of six words. 


The following result shows that the k-truncation of pop(p) is always in the 
tumble of the k-truncation of p. 


PROPOSITION 8.40 


For all permutations p, and all positive integers k, the relation 


(pop(p)) |k€ T(p |x) 
holds. 


Get Them All. Algorithms and Permutations. 371 


PROOF First we consider T(p |x). Let pi), pia), , Prey be the de- 
composition of p into maximal (nonextendible) decreasing sequences. Then 
clearly, 


P \k= (Py, Pia)» Deey) |k= (Peay) Ie (Peay) Le + (Deey) Te 


As the pim) are maximal decreasing subsequences, the last entry of p(m) is 
smaller than the first entry of pym41). Therefore, all the 10-factors of p |; 
occurs within one of the factors pym) | k. Also note that each of the k- 
truncations Pim) |x are of the form 1/0/, 1", or 0’, depending on whether the 
first and last entries of the decreasing run pi) are more than k or not. 

When constructing T(p |,), one possible choice of factors to reverse is the 
set of all factors of p(m) | k that are of the form 1'0’, since their union contains 
all 10-factors of T(p |x). 

Let us now compute pop(p) |x. We have 


pop(p) |e = pop(pi1) (p(2y ++ Pye) |e 


(Pity) Ie (eay’) Te ++ Gey’) Te 
= (pity) |e (Bia) le + Ue’) IRE T(P Ie): 


The last word is an element of T(p |x) since reversing factors pi) that are 
of the form 0/ or 14 is the same as not reversing them, and we showed in the 
preceding paragraph that if we reverse all pi) factors that are of the form 


1'07, then we get a word in T(p |g). This completes the proof. ff 


COROLLARY 8.41 
For all permutations p, and all positive integers k and t, the relation 


pop’ (p) |ké T"(p |k) 


holds. 


In other words, the k-truncation of the output of t runs through the pop 
stack can also be obtained by k-truncating the same permutation, then ap- 
plying the tumble operator to it t times. For easier parsing of the following 
proof, note that pop acts on permutations while T acts on k-truncations of 
permutations, that is, words over the alphabet {0, 1}. 


PROOF We use induction on t, the initial case of t = 1 being done in 
Proposition 8.40. For the induction step, note that 


pop’ (p) |k= (pop(pop**(p))) |k€ T ((pop**(p)) le) C T(T*"(p |x); 


where we used the induction hypothesis in the last step. | 
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For a binary word w, let F(w) denote the word obtained from w by reversing 
(”-flipping”) all 10-factors of w. For instance, if w = 011010, then F(w) = 
010101. We will call F'(w) the flip of w. It is obvious from this definition that 
F(w) € T(w) for all w. 

Now we introduce a partial order on the set of all finite binary words. If vu 
and w are two such words, then we say that v < w if for all 7, the 7th zero 
of v, if it exists, occurs in the same position or earlier than the ith zero of 
w. For instance, 00110 < 01010 in this partial order, while the words 10001 
and 01010 are incomparable. The following proposition shows that the flip 
operation preserves this partial order. 


PROPOSITION 8.42 
Ifu<w, then F(v) < F(w). 


PROOF Let vu<w. If F(v) < F(w) does not hold, then there is an i so 
that the ith 0 of F(w) occurs before the ith zero of F'(v). Crucially, flipping 
a word moves each letter of that word by at most one position to the left. So, 
it has to be the case that the ith 0 of w did move to the left, but the ith zero 
of v did not, and that originally they were both in the same position, say in 
the rth position. So, the zth zero 0 w is immediately preceded by 1, but the 
ith zero of v is not. However, that means that in the (r — 1)st position of v, 
we have the (4 — 1)st zero of v; whereas in the (r — 1)st position of w, we have 
a 1, meaning that the (4 — 1)st zero of w occurs before position r — 1. This 
contradicts the assumption that v < w. | 


Sorting a binary word means turning it into the word 071°. Note that any 
tumble of a binary word w takes each of the zeros in w at least as far left as 
F does. Therefore, the number of flips it takes to sort w is an upper bound 
for the number of tumble operations it takes to sort w. Furthermore, for any 
word w that consists of a copies of 0 and b copies of 1, the inequality w < 1°0% 
holds. Therefore, by repeated applications of Proposition 8.42, it follows that 
if m flips sort 1°0%, then m flips, and; therefore, m tumbles, sort any word w 
that consists of a copies of 0 and b copies of 1. This motivates the following 
lemma. 


LEMMA 8.43 
It takes precisely a+ b—1 flips to sort 1°0°. 


PROOF Note that throughout subsequent applications of the flip op- 
erator, zeros never leapfrog each other. So it suffices to show that it takes 
a+b—1 flips for the rightmost zero to get into its final position, which is the 
ath position. The rightmost 0 will not move during the first a — 1 flips, after 
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which it will move one notch to the left during each flip. As that last zero 
must move b notches to the left, the proof is complete. 


As we pointed out in the paragraph right before Lemma 8.43, this means 
that all binary words of length n can be sorted in n—1 tumbles. That includes 
the k-truncation of any permutation p. In other words, if w € T"~1(p |x), 
then 

w< F'l(p a) = okyrok, 


Applying Corollary 8.41 with ¢t = n— 1, we get that 
pop” *(p) |x= OF 1"-*, 


as we needed. This shows that in pop”~!(p), the entries that are at most 
k precede the entries that are larger than k, and that happens for every k. 
Therefore, pop” !(p) must be the identity permutation, and we have proved 
Theorem 8.37. 


a 


8.4 Variations of Stack-Sorting 


As we mentioned in Section 8.2, the stack sorting operation s we have been 
discussing is not the earliest one in this line of research. The first comprehen- 
sive study of stack-like sorting devices is due to Donald Ervin Knuth, [230], 
Section 2.2.1. His approach to stacks was slightly different than ours. He 
considered the following problem. 

Let us start with the permutation 123.--k, instead of ending in it, and let 
us try to obtain as many permutations from it as possible, using the stack. 
This time we do not require that the entries in the stack be in increasing order, 
as that would prevent us from obtaining any permutation different from the 
identity. Therefore, at any point of time, we have two choices. Either we put 
the entry on the top of the stack to the output, or we put the next entry of 
the input into the stack. 

If an n-permutation p can be obtained from 12---n in this way, then we 
call it obtainable. 


Example 8.44 
Let p = 231. Then p is obtainable as we can proceed as shown in Figure 8.5. 


It is then proved in [230] by a simple direct argument that p is obtainable 
if and only if it is 312-avoiding. We suggest that the reader try to find such 
a proof. On the other hand, as we have studied the stack sorting operation 
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Output Stack Input 
123 
1 23 
: 3 
1 
2 1 3 
2 3 
1 
23 1 
231 


FIGURE 8.5 
Obtaining p = 231. 


s so extensively, we will reduce this problem to the characterization of stack- 
sortable permutations. 


PROPOSITION 8.45 


The n-permutation p is obtainable if and only if p~! 


is stack sortable. 


PROOF Let p~! be stack-sortable. This means that a certain sequence 
of movements of the entries of p~! through the stack (namely, the sequence 
defined by s) turns p~! into id = 12---n. Carrying out the same sequence of 
movements on p—!-p will turn p~!-p into id-p = p. In other words, we could 
say that we relabeled the entries of p~! so that it became id, before sending 
it through the stack. So p is obtainable. 

Now let us assume that p is obtainable. That means that a certain sequence 
of movements through the stack turns id into p. Then that same sequence of 
movements turns p~! into id. Note that this implies that at any given point 
of time, the entries of the stack were in increasing order with the smallest one 
on the top. Indeed, if i < 7, and 7 had been above 7 in the stack, then 7 would 
be before i in the output, so the output could not be id. So the sequence of 
movements that turns p~! into id is precisely the stack sorting operation s, 
so s(p—') = id, and our proof is complete. 
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As a permutation is 312-avoiding if and only if its inverse is 231-avoiding, 
we see again that p is obtainable if and only if it is 312-avoiding. 

In both approaches to stacks we have seen, the crucial property of the stack 
was that the entries entered the stack at its top, and left it at its top. This was 
the property that enabled us to change the order of some entries by passing 
them through the stack. There are other sorting devices with slightly different 
restrictions. One of them is a queue, which differs from the stack because the 
entries enter at the top of the queue, and leave at the bottom of the queue. 
So the entries will leave the queue in the order they were received (as is the 
case, hopefully, with a dentist’s waiting room). Therefore, the queue is not 
an exciting sorting device as the only permutation it can sort is the identity 
permutation itself. 

Knuth has considered several generalizations of stacks and queues in his 
ground-breaking work [230]. In all of these, his approach was to look for the 
set of permutations that could be obtained by some kind of sorting device. The 
most powerful sorting device considered in [230] is the double-ended queue, or 
deque. This is a sorting device that has the capabilities of both a stack and 
a queue, and even a little bit more. That is, entries can enter at both ends 
of the deque, and they can leave at both ends of the deque. Again, there is 
no restriction on the order of the entries that are in the deque. Let us call an 
n-permutation p general deque obtainable if it can be obtained from 12---n 
using a deque. The adjective general is inserted to avoid confusion as some 
authors use the word deque for a more restricted sorting device. That is, p is 
deque-obtainable if it can be obtained from 12---n by a movement sequence, 
in each step of which we either place the next entry of the input to either end 
of the deque, or place the entry currently at the top or bottom of the deque 
into the output. 


Example 8.46 
The permutation 3412 is deque obtainable. See Figure 8.6 for a movement 
sequence leading to this permutation. 


In order to have an equivalent of Proposition 8.45, we need to define when 
we call a permutation general deque-sortable, or gd-sortable, and what the 
gd-sorting algorithm is. As the general deque is more flexible a tool than 
the stack, we can allow more flexibility in the deque sorting algorithm than 
in the stack sorting algorithm. That is, our sorting algorithm will not be 
deterministic, and the elements in the stack will not have to be in increasing 
order. They will have to form a unimodal sequence, though. The goal of the 
deque sorting algorithm is again to turn a permutation p into the permutation 
id = 12---n. We define a generic step of the gd-sorting algorithm as follows. 


Nondeterministic General Deque Sorting. 
START At any given step, if the deque is empty, we just place the next entry 
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Output Deque | Input 


Start 1234 
1 
After Steps 1-2 2 
3 
1 
After Steps 3-4 2 
4 


After Steps 5—6 34 


End 3412 


FIGURE 8.6 
Obtaining p = 3412 by a deque. 


of the input in the deque. If there is only one entry a in the deque, then we 
can place the next entry of the input on either side a in the deque. 

Otherwise, we compare the next element of the input, p;, to the entries at 
the top and bottom of the deque, x, and y. 


(a) If p; is smaller than exactly one of x or y, then p; is placed to the end 
of the deque where the element larger than p; is. 


(b) If p; < x and p; < y, then we can place p; to either end of the deque, 
and go back to START. 


(c) If p; > and p; > y, then we move the smaller of x and y to the output, 
and go back to START. 


If there are no more entries in the input, then we empty the deque, always 
moving the smaller of the two entries at the ends of the deque to the output 
END. 

Note that this algorithm assures that the entries in the deque indeed always 
form a unimodal sequence. 


DEFINITION 8.47 Ann-permutation p is called general deque sortable 
if it can be turned into id using the above nondeterministic algorithm. 


The only nondeterministic step of this algorithm is (b). Definition 8.47 
means that if there is any series of choices at step (b) that will result in the 
identity permutation at the end, then p is called gd-sortable. See Exercise 26 
for a deterministic version of this same algorithm. 
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PROPOSITION 8.48 
A permutation p is general deque obtainable if and only if p— 
sortable. 


! is general deque 


PROOF Analogous to the proof of Proposition 8.45. | 


It seems that general deque sorting is a significantly more complex pro- 
cedure than 2-stack sorting. We will see that this intuition holds true in 
an important aspect. In another aspect, however, it fails, as shown by the 
following result that was proved in a slightly different form by Pratt [264]. 


LEMMA 8.49 
The set of general deque sortable permutations forms a class. 


In other words, if g is general deque sortable, and p < q in the pattern 
containment order, then p is also general deque sortable. Recall that this was 
not true for 2-stack sortable permutations. 


PROOF — (of Lemma 8.49) As taking inverses preserves the pattern con- 
tainment relation, it is sufficient to prove that the set of general deque ob- 
tainable permutations forms a class. If g is gd-obtainable with a sequence S 
of movements, and p < q, then p is gd-obtainable by the subsequence of S 
that belongs to a given copy of p contained in gq. Indeed, the presence of the 
remaining entries of g is not necessary for the entries of that p-copy (it does 
not help them in any way) in order to move through the deque. 


In other words, gd-sortable permutations form an ideal Igq in the poset P 
of all finite permutations ordered by pattern containment, and permutations 
that are not gd-sortable form a dual ideal Jjq in P. The minimal elements of 
Ia (or any ideal, for that matter), form an antichain A. If A were finite, then 
we could characterize Iga as the set of permutations avoiding all of a finite 
number of patterns, namely the patterns in A. However, this turns out to be 
not the case. 


THEOREM 8.50 7 
The antichain of all minimal elements of Iga is infinite. 


PROOF  Sce Problem Plus6. & 


No simpler characterization is known. So even if the gd-sortable permu- 
tations form a closed class, and the 2-stack-sortable permutations do not, 
the latter are easier to characterize, and even, enumerate. It is probably not 
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surprising after this theorem that the enumeration of gd-sortable permuta- 
tions of length n is not resolved yet. However, if we take away from the very 
strong abilities of the deque, we can succeed in the analogous task. 

We will say that a deque is input-restricted if entries can only enter at the 
top of the deque, but can leave at both ends. Similarly, we say that a deque 
is output-restricted if entries can only leave at the bottom of the deque, but 
can enter at both ends. 

The reader probably senses some symmetry between the two sorting devices, 
and as we will see, the reader is right. So for a while we will restrict our 
attention to input restricted deques. In line with our previous definitions, we 
will say that an n-permutation p is ir-obtainable if we can obtain it by sending 
the identity permutation 12---n through an input-restricted deque. For the 
definition of tr-sortability, we first need to define the sorting algorithm. This 
algorithm is a somewhat simplified version of the gd-sorting algorithm, which 
was explained immediately preceding Definition 8.47. The simplification is 
that this is a deterministic algorithm. 


Deterministic IR Deque Sorting. 
START At any given step, if the deque is empty, we just place the next entry 
of the input in the deque. 

Otherwise, we compare the next element of the input, p;, to the entries at 
the top and bottom of the deque, x, and y. 


(a) If p; < x, then put p; on top of x in the deque, and go back to START. 


(b) If p; > x, and the deque is not monotone decreasing starting at the 
top, then move the smaller of x and y in the output, and go back to 
START. (If there is exactly one entry in the deque, that is considered 
a monotone decreasing deque.) 


(c) If p; > x, and the deque is monotone decreasing starting at the top, 
then put p; on top of x in the deque, and go back to START. 


If there are no more entries in the input, then we empty the deque, moving 
always the smaller of the two entries at the ends of the deque to the output. 
The output permutation is denoted ir(p). END. 


Note that this algorithm assures that the elements of the deque always 
form a unimodal sequence. Now we can call an n-permutation p ir-sortable if 
ir(p) = id. By now it should be obvious that p is ir-obtainable if and only if 
p_' is ir-sortable. 

Knuth [230] has computed the number of ir-sortable n-permutations by a 
generating function argument. His work was completed before the pattern 
avoidance explosion, that is, the bonanza of enumerative results on permuta- 
tions avoiding certain patterns. We will now show how to reduce the problem 
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of enumerating ir-sortable permutations into a problem of enumerating pat- 
tern avoiding permutations. 

Let us first try to characterize the set of ir-sortable permutations. It is 
easy to check that all permutations of length three or less are ir-sortable. The 
problems start at length four, when we cannot ir-sort 4231, and we cannot 
ir-sort 3241. This is not an accident. 


LEMMA 8.51 
If p contains 4281 or 8241, then p is not ir-sortable. 


PROOF Let us assume p contains 4231, and let a< b<c<dforma 
copy of 4231 in p. In other words, these four entries follow in the order dbca 
in p. So d will enter the deque first. If p is to be ir-sortable, then 6 has to 
enter the deque before d can leave it; otherwise, d precedes b in the output. 
Now 6 is above d, and therefore b has to leave the deque before c arrives, let 
alone before a arrives. This means that b will precede a in the output. 

Similarly, let us assume that p contains 3241, and let x < y< vu < z form 
a copy of 3241 in p. So the order of these four entries in p is vyzx. Then v 
enters the deque first. Then, if p is to be ir-sortable, y enters before v can 
leave. Now y is above v, so it has to leave the deque before z arrives, meaning 
that y will precede x in the output. 


Having seen how difficult the problem of characterizing permutations that 
are sortable by some device can be, the reader might be bracing for some 
complicated counterpart of Lemma 8.51. If this is the case, then the reader 
will be happily surprised. 


THEOREM 8.52 
The permutation p is ir-sortable if and only if it avoids both 4281 and 8241. 


PROOF As Lemma 8.51 proves the “only if” part, we only have to prove 
the “if” part. 

We prove the contrapositive. Let p be such that ir(p) 4 id. Let a < b 
be two entries that form an inversion in ir(p), that is, b precedes a in ir(p). 
It is straightforward to see that in this case, b has to precede a in p, too. 
(Recall that the deque is unimodal.) What forced b to leave the deque before 
a arrived? The only step of the sorting algorithm when an entry has to leave 
the deque is (b). Therefore, b had to leave the deque at a point when b was 
the smaller of the two entries b and c at the two ends of the deque, the deque 
was not monotone decreasing, and an entry x larger than both b and c was 
the next entry of the input. At that point, one of the following two situations 
had to occur. 
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1. The entry b was at the top of the deque, and c was at the bottom. That 
means c precedes 0 in p, and so cbza is a 3241-copy in p. 


2. The entry c was at the top of the deque, and b was at the bottom. As 
we know that the deque was not monotone decreasing, there had to be 
an entry y in the deque that was larger than c. Then y precedes c in p, 
and the subsequence ycaxa is either a 3241-pattern, or a 4231-pattern, 
depending on which one of x and y is larger. 


COROLLARY 8.53 
Let rp, be the number of ir-sortable n-permutations, with rp = 0. Then the 


equality 
1—z-—vl—-6z+2? 
by MZ = D) 
n>0 


holds. 


PROOF This follows from the solution of Exercise 9 of Chapter 4. So 
the ir-sortable permutations are enumerated by the large Schréder numbers. 


Note that this implies that rj, = ys or) Cn—i. What can be said 
about output-restricted deques? These are deques that allow entries to leave 
at the top only, but allow entries to enter at either end. If we want to sort a 
permutation p by such a deque, it is obvious that we have to keep the deque 
increasing from top to bottom, which makes it easy to construct rest of the 
sorting algorithm. The reader should try to solve Exercises 24 and 25 for the 


details. 


( 


Exercises 


1. Take a permutation of length n. Assign a direction (Left or Right) 
to each entry, such as in 273r471R5z7. Call an entry x winning if its 
neighbor in the direction given to x exists, is y, and y < x. So for 
instance, in 273R471R5z, the entries 4 and 5 are winning. 


Now consider the following algorithm. Start with the permutation p; = 
1,2,---ny. Find the largest winning entry x, and interchange it with y. 
Finally, change the direction of all entries z if z > x. Call the obtained 
new permutation po. Then apply this same procedure to pg, and call 
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10. 


11. 


the resulting permutation p3, and so on. Prove that this procedure will 
generate all n! permutations of length n in the first n! steps. 


. Which n-permutation will be generated last by the above algorithm? 


Find an algorithm to generate all n-permutations with a fixed number 
k of inversions. 


(—) Find an algorithm to generate all n-permutations with a fixed major 
index m. 


(—) Prove that all n-permutations are (n — 1)-stack-sortable. 
Is there a permutation p so that s(p) = 2413657? 


Characterize all n-permutations that are not (n — 2)-stack-sortable. 
Then enumerate these permutations. 


Characterize all n-permutations that are not (n — 3)-stack-sortable. 
Then enumerate these permutations. 


Prove that W;(n) < (t+ 1)?”. 


In the proof of Theorem 7.32 we have seen a bijection that (after taking 
reverses) maps 231-avoiding, so stack-sortable, n-permutations with k 
descents into such permutations with n — 1 —k descents. Was that 
bijection the same as the bijection f of Theorem 8.22 in the special case 
t=1? 


Consider the following modification of the t-stack sorting operation. 
Instead of passing a permutation through a stack t times, we pass it 
through t stacks placed next to each other in series as follows. The first 
stack operates as the usual stack except that when an entry x leaves 
it, it does not go to the output right away. It goes to the next stack if 
x > j, where j is the entry on the top of the next stack. If 7 < 2, then 
x cannot move until 7 does. 


The general step of this algorithm is as follows. Let S1,5S2,---,5; be 
the t stacks, with a; being the entry on top of stack S;. If the next entry 
x of the input is smaller than a;, we put x on top of S;. Otherwise, we 
find the smallest i so that a; can move to the next stack (that is, that 
a; < Qj41 or Sj41 is empty), and move a; on top of $;41. If we do not 
find such 2, or if $,,S2,--- ,S;-; and the input have all been emptied 
out, then we put the entry on the top of S; into the output. 


(a) Let ¢ = 2. What is the image of 231 under this operation? 


(b) Characterize all permutations p whose image s;(p) under the op- 
eration defined above is the identity permutation. 
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The above sorting algorithm is sometimes called left-greedy [22] as we 
always make the leftmost move possible as the entries move from the 
input on the right to the output in the left. 


Prove that W;(n) cannot be larger than the number of Standard Young 
Tableaux of shape (¢ + 1) x n in which row 7 does not contain two 
consecutive integers, for 2 <i <t. 


Prove that for any fixed n, the sequence W,-2(n, k)o<k<n-1 is log- 
concave. 


Prove that for any fixed t, and any even positive integer n, the number 
W,(n) is even. 


Define the bijection f of Definition 8.27 without using decreasing binary 
trees. 


A fall in a permutation p = p,p2---Pn is an index 7 that satisfies the 
inequalities pj_; > pj > piz4i1. A rise is an index 7 that satisfies the 
inequalities pj_1 < pj < pi41. Prove that the number of t-stack-sortable 
n-permutations with k rises is equal to the number of t-stack-sortable 
n-permutations with k falls. 


Let Prot be the poset of all finite permutations that are not t-stack- 
sortable, ordered by pattern containment. Does P,.—; contain an infi- 
nite antichain? 


Call a permutation p sorted if there is a permutation q so that s(q) = p. 
What is the maximum number of descents that a sorted n-permutation 
can have? 


Let p and qg be two n-permutations so that s(p) = pr and s(q) = qr for 
some r € S,, where pr means the product of p and r in S,,. Prove that 
the decreasing binary trees T(p) and T(q) differ only in their labels, not 
in their underlying trees. 


Is there a permutation pattern r so that if p contains r, then p cannot 
be sorted? 


For p € S(n), let 6, be the unique n-permutation so that pb, = s(p), 
where the left-hand side refers to the product in S,,. Let 


B = {bp|p € Sn}. 
How many elements does B have? 


How many permutations b € S, are there so that there exists exactly 
one p € S;, so that b = b,? 
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For which n-permutations p is s~!(p) the largest (consists of the largest 
number of permutations)? 


Define the deterministic sorting algorithm outres that uses an output- 
restricted deque. 


(a) Characterize permutations that are or-sortable. 


(b) Deduce that the number of or-sortable n-permutations is equal to 
the number of ir-sortable n-permutations. 


Define a deterministic version of the general deque sorting algorithm 
that turns the n-permutation p into 12---n if and only if p is gd-sortable. 


Let us call an n-permutation p separable if 


(a) p=1, or 
(b) p = LR, where L and R are both separable permutations (after 


relabeling), and the entries of L are either the smallest |Z| elements 
of [n], or the largest |Z] elements of [n]. 


Find a characterization of separable permutations in terms of pattern 
avoidance. 


We know that if pop(p) = 12---n, then s(p) = 12---n. However, is 
it true that if pop’(p) = 12---n for some t, then s‘(p) = 12---n also 
holds? 


Which permutations are sortable by t parallel queues? That is, we have 
t queues, the entries can only enter them at the top, and leave at the 
bottom, but for each entry of the input, we are free to decide into which 
queue that entry will enter. Once an entry leaves its queue, it goes 
immediately into the output. 


Consider the following two modifications of the sorting algorithm of 
Exercise 11. 


(a) Sorting by t stacks in series. This is an undeterministic algorithm 
defined as follows. In any given step, move one entry from the top 
of a stack $; to the top of the next stack S;41 so that the new stack 
remains increasing from top to bottom. (Recall that So is the input 
and S;41 is the output.) In other words, omit the requirement that 
in each step we have to do this with the stack of the smallest index 
i for which this is possible. 


(b) Left-greedy sorting through t stacks. That is, in each step of this 
deterministic algorithm, we find the largest index 7 so that we can 
move the entry from the top of S$; to Sj, without violating any 
constraints. That is, the $1, 52,--- ,S; all have to remain increas- 
ing from top to bottom, and the output has to be a string 123---m 
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for some m. If no such move is possible, then we say that the al- 
gorithm fails. 


Prove that if t = 2, then the algorithms defined in (a) and (b) are 
equivalent, that is, they sort the same set of permutations. 


31. Find a formula for the number of stack-sortable permutations of length 
n with exactly k peaks. Recall that a peak in a permutation is an entry 
that is larger than both of its neighbors. 


32. Let p = pip2...pPn be a permutation, and set pny, = n+l. fi 
is a descent of p, then let r;(p) be the permutation obtained from p by 
moving a; to the right as little as possible so that a; is between entries a, 
and aj+1 satisfying a; < a; < aj41. For instance, r3(314265) = 312465. 


Let d; < dz <... < dx be the descents of p. Prove that 
Td ad ace (ra, (p))) = s(p). (8.6) 


33. Find the ordinary generating function for the numbers a, of 2-stack- 
sortable n-permutations that avoid the pattern 132. 


34. Let Wi(z) = 039 Wil(n)z”. Prove that W;(z) is not a rational func- 
tion. 7 


35. A nonnegative integer m is called a fertility number (with respect to the 
stack sorting operation) if there exists a permutation p so that there are 
exactly m permutations q satisfying s(q) = p. Prove that if A and B 
are fertility numbers, then so is AB. 


a 


Problems Plus 
1. Prove that for all n and k, the equality 


(n+ k)!(2n —k—1)! 


Wan. k) = 7 yin = HI@k+ Dian Dk 1)! 


(8.7) 


holds. 


2. A {(1,0)-tree is a rooted plane tree whose vertices are labeled with 
positive integers according to the following rules. 


(a) The label of each leaf is 1. 


(b) The label of each internal node is at most the sum of the labels of 
its children. 
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FIGURE 8.7 
A (1, 0)-tree. 


(c) The label of the root is the sum of the labels of its children. 


See Figure 8.7 for an example. 


Prove that the number of 3(1,0)-trees on n+1 vertices is equal to W2(n). 


3. Prove that there exists a simplicial complex A so that the set of (k — 1)- 
dimensional faces of A is in bijection with the set of 2-stack-sortable 
n-permutations having k ascents. 


4. Sorted permutations were defined in Exercise 18. Let p be a sorted 
permutation, and denote s~‘(p) the set of its preimages under the stack 
sorting operation. Prove that there is always a permutation q € s~'(p) 
that has strictly more inversions than all other elements of s~'(p). 


5. Find an algorithm that decides whether a given permutation is sorted. 


6. Find a sufficient and necessary condition for a permutation to be gd- 
obtainable. 


7. Recall the definition of sorting by t stacks in series from Exercise 30. 
Let ¢ = 2. Characterize the set of permutations that are sortable by 
this procedure. We will call these permutations two-series-sortable. 


8. Find a formula for the number of permutations of length n that are 
two-series-sortable. 


9. Modify the sorting procedure of Exercise 11 by making it undetermin- 
istic as follows. In any given step, we can either 


(a) Put the next element of the input into any stack Sj, for any i € [¢], 
or 
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(b) Put the entire content of stack S; into the output for some j € [¢] 
(without changing the order of the entries). 


Note that the stacks S; are in fact pop-stacks in this model. This col- 
lection of pop-stacks is referred to as t pop-stacks in parallel. Let us 
call a permutation p sortable by ¢t pop-stacks in parallel if there is a 
sequence of steps of the above two kinds that turns p into the identity 
permutation. 


Is it true that the set of all permutations that are sortable by t pop- 
stacks in parallel forms a closed class? 


Let f(m) be the number of all n-permutations that are sortable by two 
pop-stacks in parallel. Prove that f(n) =n! ifn < 3, and 


f(n) = 6f(n — 1) — 10f(n — 2) + 6f(n — 3) 
ifn > 3. 


In part (b) of Exercise 30, we defined the left-greedy algorithm. The 
right-greedy algorithm is just the algorithm defined in Section 8.2.2.1. 
Is it true that if a permutation is sortable by the right-greedy algorithm 
on t stacks, then it is also sortable by the left-greedy algorithm on t 
stacks? 


(a) Let B,(n, &) be the number of t-stack-sortable n-permutations that 
have exactly k peaks. Prove that 


n/2 B,(n, k) 


Wail?) =) panaaae tay, (8.8) 


k=0 

where W,,:(z) was defined in (8.5) as the generating polynomial 
of t-stack-sortable n-permutations with respect to their number of 
descents. 


(b) Deduce Theorem 8.30. 


Prove that for t= 1, t = 2, and t = n — 2, the polynomial W,, 4(z) has 
real roots only. 


Let us call a permutation uniquely sorted if it is sorted and has exactly 
one preimage under the stack sorting operation. Find a sufficient and 
necessary condition for p to be uniquely sorted. 


We defined fertility numbers in Exercise 35 of this chapter. Are all 
nonnegative integers fertility numbers with respect to the stack sorting 
operation? 
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(a) Prove that if n > 4 is even, then n points in the plane that are 
not all collinear determine at least n different directions, and this 
bound is sharp. 


(b) Prove that if n > 4 is even, then n +1 points in the plane that are 
not all collinear determine at least n different directions, and this 
bound is sharp. 


Let p be any permutation. What can we say about the relation between 
|s~*(p)| and |s~*(s(p))|? 


If s‘(p) is the identity permutation, but s‘~1(p) is not, then we say that 
the depth of p in the stack sorting tree is t. Let D(n) be the average 
depth of all permutations of length n. So D(1) = 0, D(2) = 1/2, and 
D(3) = 1. What can we say about the fraction D(n)/n as n goes to 
infinity? 


Let us say that permutation p is t-sorted if there exists a permutation 
m so that s‘(7) = p. At most how many descents can a t-sorted permu- 
tation of length n have? 


Let us assume that n and ¢ are of the same parity. Characterize all 
t-sorted permutations of length n that have a maximum number of de- 
scents. Then enumerate such permutations. 


a 


Solutions to Problems Plus 


1. 


We have mentioned in the text that 2-stack sortable n-permutations are 
in bijection with nonseparable rooted planar maps on n+ 1 edges. That 
bijection, which can be found in [149], turns the number of descents of 
the permutation into the number of vertices of the map, and turns the 
number of ascents of the permutation into the number of faces of the 
map. However, rooted nonseparable planar maps have been enumerated 
according to their number of vertices and faces in [101], and have been 
found to be counted by the expression on the right-hand side of (8.7). 
An alternative solution will be mentioned in the solution of the next 
Problem Plus. 


See [128] for a short proof. Also note that 6(1,0)-trees on n +1 vertices 
having & leaves are in bijection with 2-stack-sortable n-permutations 
having & descents. As the former have been enumerated in [127] and 
have been shown to be counted by (8.7), this provides an alternative 
proof for the previous Problem Plus. 
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3. This result was published in [56]; see that paper for the details that are 


omitted here. Denote by ree the set of all 6(1,0)-trees on n+ 1 


nodes having & internal nodes. Our plan is as follows. To each 6(1,0)- 
tree T € roe we will associate a k-tuple (Ti, T2,--- , Th) € Tee 


of B(1,0)-trees —, in an injective way. By the solution of the previous 
Problem Plus, this is equivalent to the statement to be proved. 


First, we specify the order in which we will treat the & internal nodes 
of T. To that end, we extend the notion of postorder from binary plane 
trees to plane trees the obvious way. That is, for each node, read its 
subtrees from left to right, then the node itself, and do this recursively 
for all nodes. This rule linearly orders all nodes of T’, and in particular, 
turns our set {Vi,--- , V;} of internal nodes into the k-tuple (Vi,--- , Vi) 
of internal nodes. 


Let i € [k] and let V; be the ith internal node of our ((1,0)-tree T. 
Let V; have d; descendents, excluding itself. Denote by J; the number 
of nodes of T that precede V; in the postorder reading of T’. Similarly, 
denote by r; the number of nodes of T that follow V; in the postorder 
reading of T. 


Define T; as the unique {(1,0)-tree with one internal node Z; so that 
Z;, has d; descendents, and the root of T; has ¢; leaf-children on the left 
of Z; and r; leaf-children on the right of Z;. The only node whose label 
has to be defined is the only internal node Z;, and we set labelr,(Z;) = 
labelr(V;). 


We show that we can indeed always set labelr,(Z;) = labelr(V;), that is, 
labelr(V;) is never too big for the label of Z;. Indeed, Z; has d; children, 
all leaves, so any positive integer at most as large as d; is a valid choice 
for the label of Z;. On the other hand, V; has d; descendents in T, so 
labelr(V;) < d;, and therefore labelr(V;) is a valid choice for labely,(Z;). 
Our decomposition map h is then defined by h(T) = (T1,T2,--- , Tx). 
See Figure 8.8 for an example of this map. 


One can then show by induction on k that the map 


1,0 1,0 
he DEV, > (Dacial 


defined by h(T) = (11, T2,--+ , T,) is an injection. 
Note that h is not a surjection. Indeed, for (T1,7T>,--- ,T,) to have 
a preimage, we must have ¢; < 2 <--- < &. This also shows that 


k-tuples of 6(1,0)-trees with one internal node, and k-element sets of 
G(1,0)-trees with one internal node are equivalent for our purposes. 


Finally, we show that the defining property of simplicial complexes 
holds for our construction. That is, we show that if there exists a 
B(1,0)-tree T so that A(T) = {T1,To,---,T,}, then for any subset 
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FIGURE 8.8 
Decomposing a (1, 0)-tree. 


I = {t1,%2,--- ,i;} C [k], there exists a 6(1,0)-tree T; so that h(Ty) = 
aaa ag aes elas ja 


It is clear that all integers ¢;,,¢,,--- ,¢:, are different as even the in- 
tegers ¢),€2,--- ,& are all different. Relabel the elements of J so that 
they are in increasing order. This gives us a j-tuple (Tj,,7;,,--+ ,Ti;) 


of 6(1,0)-trees with one internal node. 


Let us assume first that J has k — 1 elements, with 7 being the missing 
element. Then we construct 7; from T as follows. Let V; be the ith 
internal node of JT’. Remove V; from 7’, and connect all its children to 
the father F; of V;, preserving their left-to-right order. Say the rightmost 
child of V; was R;. Now add V; back to the tree so that it is a child of 
F, and it immediately follows R; in the left-to-right list of the children 
of F;. Note that now V; became a sibling of one of its former children, 
and it became a leaf. Therefore, we have to change the label of V; to 
1, but we do not have to change any other labels. Indeed, there is no 
other node who lost descendents in this operation. (The only exception 
is when V; was a child of the root. In that case, we may have to change 
the label of the root, but that will not cause any problems.) Call the 
8(1,0)-tree obtained this way T;. See Figure 8.9 for an example of this 
procedure. 


It is straightforward to verify that h(T7) = (Ti,,Ti.,°°:,Tiy,_,). UI 
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FIGURE 8.9 
Removing an internal node. 


has less than k — 1 elements, then we construct T; by iterating this 
procedure. 


4. This result was proved by M. Bousquet-Mélou, and can be found in 
[88]. She defines two operations on decreasing binary trees that increase 
the number of inversions in the corresponding permutation, but do not 
change the postorder reading of the tree. One operation concerns ver- 
tices that have a left subtree only, and turns that subtree into a right 
subtree. The second operation concerns vertices v that have a left child 
a and a nonempty right subtree T so that the leftmost vertex y of T 
is larger than a. Then the operation takes the subtree rooted at a and 
moves it so that it is the right subtree of y. See Figure 8.10 for an 
example. 


The author then proves by induction on the size of the trees that these 
operations will always lead to a unique tree Ty¢,. She calls that permu- 
tation corresponding to that tree a canonical permutation. Note that 
neither of the two operations defined above can be carried out on Tyan, 
which shows some properties of this tree. 


5. A recursive algorithm to decide this question was developed by M. 
Bousquet-Mélou [87]. She uses a decomposition first mentioned in [338]. 
Let p be an n-permutation that has k — 1 descents, or, in other words, 
k ascending runs. Let these ascending runs be r1,1r2,--- ,r~. We have 
seen in the text that taking the stack sorted image of a permutation 
q is nothing else but taking the decreasing binary tree T(q) of q and 
then reading it in postorder. Now let us assume that s(q) = p. Then 
any time we pass in p from one ascending run to the next, in the pos- 
torder reading of T,, we must jump from the left subtree of a vertex to 
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Finding the canonical preimage of 21346578. 


the right subtree of a vertex, as that is the only way we can obtain a 
descent. This observation justifies the following algorithm. 


To obtain a permutation q so that s(q) = p, take the ascending run 
decomposition 71,72,:-:,7T~ of p, and for each 7, take the decreasing 
binary tree that is corresponding to the reverse of r;. Denote these 
trees by T;, for i € [k]. Attach a left-half edge to the top of these trees. 
Now try to build up q right to left as follows. In Step 1, find the smallest 
element in r, that is larger than the largest element in r,_1, and attach 
Tr-1 to that vertex as a left subtree. If there is no such element (which 
would mean that p did not end in n), then p is not sorted. Otherwise, 
continue this way. In step 7, find the smallest element in the leftmost 
path of the tree already created that has no left child and is larger 
than the largest element of T;,_;. If there is no such element, then the 
previous paragraph implies that p is not sorted. Otherwise, we continue 
this procedure till an n-vertex tree is created. In that case, p is sorted 
as p = s(q), where the permutation corresponding to the created tree is 
q. Note that the permutation q will be the canonical preimage of p. See 
Figure 8.11 for an example. 


6. The following was proved in [264] in a slightly different form. The set 
of minimal non-gd-sortable permutations is a; = 52341, ag = 5274163, 
ag =729416385,a, =9211416385 10 7, and so on, and 
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FIGURE 8.11 
Finding a preimage for p = 213547689. 


10. 


all permutations that can be obtained from the b; by interchanging the 
two maximal entries, and/or interchanging the first two entries. Note in 
particular that in the a;, the even entries are fixed points and the odd 
entries are cyclically translated (except in a1). 


. It is proved in [22] that these are the permutations that avoid all patterns 


of the form gm = 2(2m — 1)416385---2m(2m — 3), for each m > 2. In 
other words, in position 22 — 1, the pattern q,, contains the entry 2:2, 
and in position 27 the pattern q,, contains the entry 27 — 3 if i > 1, and 
the entry 2m — 1 is in the second position of qm. 


. It is proved in [22] that the number A(n) of these permutations is equal 


to Av, (1342). We have proved an exact formula for Av,,(1342) in The- 
orem 4.56. 


. Yes. In fact, more is true. It is proved in [21] that for any t, there exists 


a finite set F; of permutation patterns so that p is sortable by t pop- 
stacks in parallel if and only if p avoids all patterns in F;. So the class 
of these permutations is not just an ideal, but an ideal generated by a 
finite number of elements. The proof is constructive, so the elements of 
F, can actually be found. For t = 2, there are seven of them. 


This result is due to Michael Atkinson [21]. 
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Yes, that is true as was proved by Rebecca Smith [287]. 


(a) That result is due to Petter Brandén [94], who proved that the 
descent generating polynomial of certain classes of permutations 
has a non-negative expansion in the basis B = {z*(1+ z)"t1-?*}, 
of polynomials, where k = 0,1,--- ,|n — 1/2]. It turns out that 
t-stack sortable permutations are one example of the classes for 
which Brandén’s results hold. 


(b) This follows from the existence of the expansion explained above, 
since all the elements of the basis B are symmetric and unimodal 
polynomials that have the same center of symmetry. 


The special cases of t = 2 and t = n— 2 were proved by Petter Brandén 
in [93] who built his own powerful machinery to prove the real zeros 
property for a much larger class of polynomials. The special case of 
t = 1 is implicit in the works of Francesco Brenti, but an explicit proof 
can be found in [91]. 


A permutation of length n is uniquely sorted if and only if it is sorted 
and has exactly (n — 1)/2 descents. This is a result of Colin Defant 
[134]. 


No. Colin Defant [135] showed that the smallest fertility number that 
is congruent to 3 modulo 4 is 27. 


These questions were the motivation for the combinatorial geometer 
Peter Ungar that somewhat surpisingly lead to an argument proving 
Theorem 8.37. See [314] for proofs. 


It is proved in [137] that |s~+(p)| < |s~!(s(p))|. 
It is proved in [137] that 
0.62433 ~  < liminf D(n)/n < limsup D(n)/n 


< 0.6(7 — 8 log 2) ~ 0.87289. 


Here 2 is the Golomb-Dickman constant , which is 1/n times the ex- 
pected length of the longest cycle of a permutation of length n selected 
uniformly at random out of all permutations of length n. 


Colin Defant [136] proved that this number is at most [(n — t)/2]. 


Colin Defant [136] proved that such a permutation p = pip2-+: Dy is t- 
sorted and has [(n—t)/2] descents if and only if its left-to-right maxima 
are P1,P3,P5,°**Pn—t+1Pn—t+2°**Pn- Then it follows that p; = 7 if 
j>n-—t+1. From this, it is easy to prove by induction on n — t that 
the requested number is (n — t — 1)!!. 


Taylor & Francis 
Taylor & Francis Group 
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How Did We Get Here? Permutations as 
Genome Rearrangements. 


9.1 Introduction 


Let p and q be two permutations, written in one-line notation. In this section, 
we will attempt to transform p into q using only certain kinds of allowed steps. 
If k is the smallest integer so that p can be transformed into q using k allowed 
steps, then we say that the distance d(p,q) of p and q is k. 

The motivation for this research comes from molecular biology, where genomes 
of various species can be modeled by permutations, and the distance between 
these permutations can be interpreted as the evolutionary distance between 
species. A classic paper of the field, by Shridhar Hannenhalli and Pavel 
Pevzner [211], is entitled Transforming Cabbage into Turnip, which in turn 
explains the cover page of this book, since the genes constituting the genomes 
are modeled by the entries of the permutations corresponding to the genomes. 

The notions of distance we study in this book will all be left-invariant, that 
is, they will have the property that for any n-permutations p, q, and r, 


d(p,q) = d(rop,roq). (9.1) 


Here ro p means the composition of the bijective functions r : [n] > [n] and 
p: [n| > [n], with r applied first. The left-invariant property described in 
(9.1) is explained by the fact that the distance between two genomes depends 
only on those two genomes, and not on the labeling of the genes in each 
genome. So relabeling the genes in one genome, and then relabeling the genes 
of the other genome in the same way will not change the distance between 
the two genomes. 

Setting r = q~+, formula (9.1) means that d(p,q) = d(q~! 0 p,id). So if 
we know how to determine the distance of a generic permutation from the 
identity, then we also know how to find the distance of any two permutations 
from each other. 

This reduces our distance-measuring problem to a sorting problem. The 
question we attempt to answer is now the following. Let us assume that the 
notion of distance is fixed. Given an arbitrary permutation p of length n, 
how many moves (of a specified kind) are necessary to transform p into the 
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increasing permutation 123---n? What is the permutation that is the most 
difficult to sort, that is, whose distance from the increasing permutation is 
the largest? How efficient is a given sorting method for the average permuta- 
tion? Can we find an algorithm that finds the shortest distance from p to the 
increasing permutation? 


9.2 Block Transpositions 


Perhaps the most well-studied biologically motivated sorting operation is that 
of block transpositions. In a block transposition, two consecutive, adjacent 
blocks (substrings) of entries are interchanged so that the order of entries 
within each block is unchanged. Formally speaking, the permutation 


P= P1p2°+* PiPit1°** P7Pj41°°*PkPk4+1°°° Pn (9.2) 


is turned into the permutation 


qd = Pip2:** PiPjt+1°** PkPit1°** PJPk41°°*Pn- (9.3) 


Example 9.1 
Below is a block transposition with 7 = 2, 7 = 4 and k = 7. 


245613987 —+ 24 | 56 | 39 | 87 —> 24 | 139 | 56 | 87. 


U 


We point out that this use of the word transposition is different from its use 
in group theory. 

Note that one can view a block transposition as an operation that inter- 
changes two adjacent blocks, or as an operation that takes one block and 
places it somewhere else in the permutation. 

Let t be the smallest positive integer so that there exists a sequence of t 
block transpositions that turn p into g. We then say that t = btd(p,q) is 
the block transposition distance of the n-permutations p and q. The reader 
is invited to verify that the block transposition distance is a metric, that is, 
btd(p, p) = 0 for all p, and btd(p,q) = btd(q,p) for all p and gq (since the 
inverse of a block transposition is also a block transposition), and for all p, q 
and r, the triangle inequality 


btd(p, q) + btd(q,r) > btd(p, r) 


holds. 
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We are now going to formally prove a simple but important property of the 
block transposition distance that we mentioned in the previous section. 


PROPOSITION 9.2 
The block transposition distance of two permutations is left-invariant. That 
is, if p,q, andr are n-permutations, then 


t = btd(p, q) = btd(ro p,roq). 
PROOF _ Induction on k, the case of t = 0 being trivial, and the case of 
t = 1 being easy since if btd(p,q) = 1, then there exist indices 7, 7, and k so 


that q is obtained from p as shown in (9.2) and (9.3). However, in that case, 
the same block transposition will turn 


Tro p = Pri Pro oe ‘Pr; Prist oe *PrjPrjsi oe *PrpPrigr oe ‘Pry 


into 


ro q =, PriPro _ ‘Pr, Prj41 tan ‘PrypPrist 2m *Prj Presi Te ‘Pry: 


Now let us assume that the statement is true for t and prove it for t + 1. 
Let btd(p,q) = t+ 1, and let gq’ be a permutation so that btd(p,q’) = t and 
btd(q’,q) = 1. Then, by our induction hypothesis, btd(ro p,roq’) = t and 
btd(r o q’,r oq) = 1, so there is a sequence of t + 1 block transpositions that 
take rop to rog. This proves that btd(ro p,roq) < t+1 = btd(p,q). 

In order to prove that btd(ro p,ro q) > btd(p,q), repeat this argument 
with the roles reversed, (that is, p replaced by ro p and q replaced by ro q), 
and noting that p=r-to(rop) andq=r'o(roq). 


With a slight abuse of language, let btd(p) := btd(p, id), the block transpo- 
sition distance of p, be the smallest integer k so that there exists a sequence 
of & block transpositions that takes p into the identity permutation. 

As stated in the introduction of this chapter, our goal is to find good esti- 
mates for btd(p), as well as td(n) = maxpes,, btd(p). In other words, we would 
like to find the smallest number m so that all n-permutations can be sorted 
by at most m block transpositions. It would also be interesting to find out 
which permutations are the most difficult to sort. Note that td(n) is called the 
block transposition diameter of S;,, since that number is the longest distance 
between two points in the graph whose vertices are the elements of S,,, and in 
which two vertices p and p’ are adjacent if there is a block transposition that 
turns p into p’. 

The following simple observation provides a lower bound on the block trans- 
position distance btd(p) of a permutation from the identity. 
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PROPOSITION 9.3 
No block transposition can decrease the number of descents of a permutation 
by more than two. 

In other words, if p has d(p) descents, then btd(p) > d(p)/2. 


PROOF Keeping the notation of (9.2) and (9.3), we see that the only 
positions in which a descent could be turned into an ascent by a block trans- 
position are 7, 7, and k. In order for the number of descents to decrease by 
three, such a change would have to occur in each of those three positions. 
That could only happen if all of the inequalities 


1l. pi > Di41, 
2. pj > Pj+is 
- Dk > Pk+1; 


3 
4. pi < Pj4i, 


5. Pk < Pi+1; 
6. pj < Pr+i 


held. That is impossible, since that would imply that 
Pi > Pitt > Pk > Pkt > Pj > Pj+1 > Vi- 


Based on this observation, and our intuition, it would be plausible to think 
that the decreasing permutation n---21 is perhaps the most difficult to sort, 
since it has the highest possible number of descents. This motivates the precise 
computation of btd(n--- 21). 


LEMMA 9.4 

[167] For all integers n > 3, the equality btd(n---21) = [(n+1)/2] holds. 
In particular, the block transposition diameter td(n) of the symmetric group 

Sy, satisfies the inequality td(n) > [(n+1)/2]. 


PROOF First, we show that [(n+1)/2] block transpositions are necessary 
if we want to transform the decreasing n-permutation into the increasing one. 
The decreasing permutation has n — 1 descents. The first block transposition 
away from the decreasing permutation will decrease the number of descents 
by exactly one, and the last block transposition arriving at the identity will 
also decrease the number of descents by exactly one. Hence, the other block 
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transpositions are responsible for decreasing the number of descents by n — 3, 
and Proposition 9.3 shows that that takes at least (n—3)/2 additional moves. 

Now we show that [(n + 1)/2] block transpositions are sufficient to trans- 
form the decreasing n-permutation into the increasing one. First, note that 
it suffices to prove this claim for odd n. Indeed, if n is even, we can sort the 
decreasing permutation n(n — 1)---32 using [(n + 1)/2] — 1 block transposi- 
tions, then use the remaining block transposition to move the entry 1 to the 
front. 

So we can assume without loss of generality that n = 2k+1. Then let 
the first block transposition move the two-element block (& + 1)k to the front 
of the decreasing permutation, let the second block transposition move the 
two-element block (k + 2)(k — 1) into the middle of the block that last moved 
to the front, and so on. After k block transpositions of this kind, we obtain 
the permutation 


(k +1) (k+2)-+-2k 12---k (2b +1), 


which we then turn into the identity permutation by interchanging the block 
of the first & entries and the block of the next k entries. 


Example 9.5 
We can sort p = 7654321 in four block transpositions as follows. 


1. 765 | 43 | 21, 
2. 4376 | 52] 1, 
3. 45237 | 61, 
4. 456 | 123 | 7, 
5. 1234567. 


0 


The best known upper bound for td(n) is given by the following theorem 
of Henrik Eriksson, Kimmo Eriksson, Johan Karlander, Lars Svenson, and 
Johan Wastlund. 


THEOREM 9.6 
[167] Let n > 9, and let p be an n-permutation. Then 


btd(p) < [(2n — 2)/3]. 
Therefore, td(n) < |(2n — 2)/3]. 


The proof of this theorem will be given in Problem Plus 9.6. We point out 
that if n < 9, then the slightly weaker statement btd(p) < [2n/3] holds. 
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Theorem 9.6 and Lemma 9.4 imply that 
[(m + 1)/2] < td(m) < [(2n — 2)/3], 


where the lower bound is assured by the decreasing permutation. That, of 
course, does not mean that the decreasing permutation is indeed the most 
difficult to sort, but for several years, no n-permutation p was known to 
satisfy btd(p) > btd(n---21). That changed when in 2004 Isaac Elias and 
Tzvika Hartman [157] presented the following result. 


PROPOSITION 9.7 
Let i be a nonnegative integer. Then the permutation 


EH(t)=432151312---61417 16 15---14+4+4¢ 17+ 42 16 + 4¢ 154 4% 


has block transposition distance 10 + 2%. 


In other words, EF H(i) consists of the following parts, in the following order 
left to right. 


1. The decreasing sequence 4321, 
2. the fixed point 5, 
3. the decreasing sequence of consecutive integers from 13 to 6, 


4. the repeated segments 14+ 9 17+ 7 16+ 7 15+) for all integers 7 
satisfying 0 <j <1. 


As EH(t) is of length n = 17 + 4%, we see that btd(EH(i)) = 10 + 2i > 
[(n + 1)/2] =9+4 2i. So the decreasing permutation is not the most difficult 
one to sort by block transpositions; EH (i) is more difficult. Note that this 
construction exists ifn > 17 and n = 4k +1 for some integer k > 4. 


a 


9.3. Block Interchanges 


Let p = pypo+--pn be a permutation. A block interchange is an operation 
that interchanges two blocks of consecutive entries without changing the or- 
der of entries within each block. Unlike in block transpositions, the two 
blocks do not need to be adjacent. Interchanging the blocks p;pi4i--- pita 
and pj;pj41°**Pj+o With 1 +a <j results in the permutation 


Pl‘ *Pi-1 | D3P541 °° Dj+o | Pitasi-** Dj 1 | Deena ++ Deve | Dj4041 °° Pn- 
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For instance, if p = 3417562, then interchanging the block of the first two 
entries with the block of the last three entries results in the permutation 
5621734. 

Let bid(p, q) denote the block interchange distance of p and q, that is, the 
smallest positive integer t so that there is a sequence of t block interchanges 
that takes p into q. 

The reader is invited to verify that the block interchange distance is a metric 
and that it is left-invariant. 

With a slight abuse of language, let bid(p) denote the block interchange 
distance of p from the identity permutation, that is, the number of block 
interchanges that are necessary to sort p. 

It turns out that sorting by block interchanges is much better understood 
than sorting by block transpositions. The following definition is crucial to 
understanding the strong results related to sorting by block interchanges, and 
it turns out to be useful in studying block transposition distances as well. 


DEFINITION 9.8 Let p= pip2:--pn be an n-permutation. The cycle 
graph G(p) of p is a directed graph on vertex set {0,1,---,n,n +1} and 
2n+ 2 edges that are colored either black or gray as follows. Set po = 0, and 
Poti =nt+i. 


1. Fori<i<n-+l, there is a black edge from p; to pj_1, and 


2. ForO<i<n, there is a gray edge fromi toi+1. 


The cycle graph was defined by Vineet Bafna and Pavel Pevzner in [27]. 
It has many variations, which are equivalent for most purposes, but some of 
which are easier to use for a given problem than others. 

It is straightforward to show that G(p) has a unique decomposition into 
edge-disjoint directed cycles in which the colors of the edges alternate. Indeed, 
note that each vertex 1, 2,--- ,n has one edge of each color leaving that vertex, 
and one edge of each color entering that vertex. There is a unique way for 
a cycle to leave the vertex 0, namely by going to 1 using a gray edge. After 
that, there is a unique way to leave 1 by a black edge, namely by going to 
the entry immediately preceding the entry 1 in p, and so on. When the first 
alternating cycle is completed, we disregard its edges, and find the subsequent 
alternating cycles of G(p) similarly. 

For the rest of this chapter, when we say “cycles of G(p)”, we always mean 
the alternating cycles in the alternating cycle decomposition of G(p). Note 
that a vertex can occur twice in the same cycle of G(p), if it is first visited by 
an edge of a certain color, then by an edge of the other color. 

Let c(G(p)) be the number of directed cycles in this decomposition of G(p). 
Note that c(G(p)) is not equal to the number of cycles of p in the traditional 
sense that was discussed in Chapter 3. In order to avoid confusion, we will 
use the notation c([‘(p)) for that notion. That is, c(['(p)) is the number of 
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cycles of p in the traditional sense. In other words, c(I'(p)) is the number of 
cycles in the directed graph I'(p) on vertex set [n] that has n edges, one edge 
from i to p; for each 7. Note that G(p) may have as many as n+ 1 alternating 
cycles, but ['(p) can have at most n cycles. Also note that G(p) has 2n + 2 
edges, while I'(p) has n edges. The edges of G(p) are of two kinds, and the 
alternating cycles of G(p) are alternating between edges of the two kinds. 

At this point, the reader is advised to try to solve the simple Exercises 1 
and 2 that concern c(G(p)) in the case when p is the increasing or decreasing 
permutation. 

See Figures 9.1 and 9.2 for three examples. Black edges are represented by 
solid, thick lines, and gray edges are represented by thin, dotted lines. 


G(p) GO) 


p=123 p=4213 


FIGURE 9.1 
The graphs G(p) for p = 123 and p = 4213. One sees that c(G(123)) = 4 and 
e(G(4213)) = 1. 


The following result of D. A. Christie [116] explains the importance of cycle 
graphs. 


THEOREM 9.9 
The number of block interchanges needed to sort the n-permutation p is 


n+ 1 -—c(G(p)) . 


bid(p) = 5 


Note that Theorem 9.9 implies that n + 1 and c(G(p)) are always of the 
same parity. 

Since c(G(p)) > 1 for all non-empty permutations p, Theorem 9.9 implies 
that the diameter of S,, with respect to the block interchange distance is at 
most [n/2]. 

In order to prove Theorem 9.9, we need two lemmas, the first of which is 
the following. 
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G(p) 


oe e ¢ ee @ 
O ae 1 


p=4312 is er 


FIGURE 9.2 
The graph G(p) for p = 4312. One sees that c(G(4312)) = 3. 


LEMMA 9.10 
Let p be any n-permutation that is not the identity permutation. Then there 
exists a block interchange that increases c(G(p)) by two. 


PROOF  Asp=pip2--- pn is not the identity permutation, p contains at 
least one inversion, that is, there are two indices i and j in [n] so that p; > p,; 
while 7 < 7. If there are several such pairs, then choose the pair for which 
p,; is minimal, and if there are still several eligible pairs, then choose the one 
for which p; is maximal. For easier notation, set x = p; and y = p;. It then 
follows from our definitions that x — 1 is on the left of y, and y+ 1 is on the 
right of x. (It is possible that « = 1, but then imagining the entry x —-1=0 
in front of p will not cause any problems, because, as we will see below, x — 1 
will not actually be part of the blocks to be interchanged.) 

So p has the form 


p= alley] es+ [ares [yt lee. 


Now apply the block interchange indicated by the vertical bars, that is, inter- 
change the block starting immediately after s—1 and ending in y and the block 
starting in x and ending immediately before y + 1, to get the permutation 

i, 


p =---¢-1| ce | ytl---. 


Let us call this the canonical block interchange of p. We claim that c(G(p’)) = 
e(G(p)) +2. 

In order to prove this claim, note the block interchange defined above does 
not change any gray edges of G(p), and changes either three or four black 
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edges of G(p), depending on whether y and x were in adjacent positions in 
p or not. We are going to show that the claim holds in both cases. We will 
repeatedly use the simple fact that if in G(p) there is an alternating path from 
u to v, then u and v are part of the same alternating cycle of G(p). You are 
asked to prove this fact in Exercise 7. 


1. Let us first assume that x immediately follows y in p. That means that 
the alternating cycle decomposition of G(p) contains the alternating 
path (starting with a black edge) a x—12 y y+1 6, where a is the 
entry immediately on the right of x — 1, and b is the entry immediately 
on the left of y+1 in p. In particular, those six entries were in the same 
cycle C. However, in p’, the two-vertex cycles x — 1 x and y y+1 are 
formed, and the rest of C forms a separate cycle, with the newly created 
black edge ab closing the cycle. So the cycle C of G(p) is replaced by 
three cycles in G(p’), while the other cycles are unchanged. See Figure 
9.3 for an illustration. 


FIGURE 9.3 


The canonical block interchange of p when y and x are adjacent. The arrow 
from b to a represents an alternating path, not necessarily an edge. 


2. Let us now assume that x and y are not neighbors in p, and introduce 
the notation a, b, c, and d for certain entries of p, so that x —1 a, y b, 
cx, and d y+ 1 are adjacent pairs of entries in p. Using that notation, 
G(p) will always contain the alternating path P, =a «-— 142 cand the 
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alternating path P, = by y+1d, but P, and P, may or may not be in 
the same cycle. 


(a) If P, and P, are part of the same alternating cycle Cay of G(p), 
that is, when there is an alternating path Aga from d to a ending in 
a gray edge, then in G(p’), the cycle Ca, will be replaced by three 
cycles, namely the two-vertex cycles x — 1 a and y y+ 1, and the 
cycle that contains b, d, the path Aga, and c. Other cycles will not 
change. See Figure 9.4 for an illustration. 


oS a , ; an. 
x-l x d b c a y yt+l 


c) 


P 


FIGURE 9.4 
The canonical block interchange of p when y and x are not adjacent, but are 
part of the same cycle of G(p). 


(b) If P, and P, are part of two distinct cycles C, and Cy of G(p), then 
in G(p’), each of these cycles will break up into two cycles. Indeed, 
C breaks up into the two-vertex cycle x — 1 x and the rest of Cg, 
completed by the new black edge ac, while Cy breaks up into the 
two-vertex cycle y y+ 1 and the rest of Cy, completed by the new 
black edge bd. So in this case, two cycles of G(p) turn into four 
cycles of G(p’). See Figure 9.5 for an illustration. 


A remarkable property of block interchanges is that while they can be used 
to increase c(G(p)) by two, they will never increase c(G(p)) by more than two. 
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ee aN >. z oS 
x-l x d b c a y yell 


c) 


Pp 


FIGURE 9.5 
The canonical block interchange of p when y and x are not adjacent, and are 
not part of the same cycle of G(p). 


LEMMA 9.11 
No block interchange increases c(G(p)) by more than two. 


PROOF _ A block interchange B changes at most four black edges of G(p) 
(the black edges connecting the interchanged blocks to non-moving blocks). 
Before B is executed, those black edges are part of at least one cycle, and 
after B is executed, they are part of at most four cycles. So the only way 
that B could possibly increase the number of cycles of G(p) by more than two 
would be if B broke one cycle into four smaller cycles. That would mean that 
the inverse transformation B~! of B, which is also a block interchange, would 
change one black edge from each of four different cycles, and would turn those 
four cycles into one cycle. 

However, that is impossible. Indeed, consider the block interchange, shown 
in Figure 9.6 that changes four black edges, namely the black edges aa’, b’b, 
cc’ and d'd, and assume that in B~'(p) = p’, the four new black edges, that 
is, ca’, b’d, ac’, and d’b are in the same cycle C. 

Let us start walking around C starting with ca’. What is the first newly 
created edge we encounter? It must be ac’. Indeed, if it were b/d, that would 
mean that the edges aa’ and b’b were in the same cycle of p, and if it were 
d'b, that would mean that the edges aa’ and d'd were in the same cycle of p. 
So the first new edge we encounter in C after ca’ is ac’. The next new edge 
we encounter cannot be b'd, since that would imply that the edges cc’ and b/b 
were in the same cycle of p. It cannot be d’b either, since that would imply 
that the edges cc’ and d’d were in the same cycle of p. This contradiction 
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, b b’ c c d 


a a d’ 


FIGURE 9.6 
The four new black edges cannot all be part of the same cycle. 


proves that the four new edges of p’ cannot all be part of the same cycle. 
Therefore, as block interchanges are reversible, block interchanges that turn 
some cycles into four different cycles must start with two cycles, not one. 


Now the proof of Theorem 9.9 is immediate. 


PROOF (of Theorem 9.9) The cycle graph of the identity permutation 
has n+ 1 cycles, and as we mentioned (see Exercise 2), the identity is the only 
permutation with that property. By Lemma 9.11, bid(p) > (n+1—c(G(p))/2, 
and by Lemma 9.10, bid(p) < (n + 1 — c(G(p))/2, since that many block 
interchanges can transform p into a permutation with n+ 1 cycles in its cycle 
graph, that is, into the identity permutation. 


9.3.1 Average Number of Block Interchanges Needed to Sort 
Pp 


Now that we know how to sort any given permutation by block interchanges, 
we can address the more global question of how efficient sorting by block 
interchanges is. Theorem 9.9 tells us that every n-permutation can be sorted 
by at most |n/2] block interchanges, but what can we say about the average 
permutation? 

It follows from Theorem 9.9 that we could answer this question if we could 
count all n-permutations whose cycle graph has a given number of alternating 
cycles. This motivates the following definition. 


DEFINITION 9.12 The number of n-permutations p satisfying c(G(p)) = 
k is called a Hultman number, and is denoted by Sxy(n,k). 
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The name Hultman numbers was given by Anthony Labarre [148] and honors 
Axel Hultman. 


Example 9.13 

For n = 3, we have Sy (3,4) = 1 and Sq (3, 2) = 5. Indeed, all 3-permutations 
except the identity have two alternating cycles in their cycle graph (because 
they have block interchange distance 1). 


It turns out that for a permutation p, there is a rather close, but nontrivial, 
connection between c(G(p)), which is the number of alternating cycles of 
the cycle graph of p, and c([(p)), which is the number of cycles of p in the 
traditional sense. The following result of Doignon and Labarre [148] makes 
this explicit. 


THEOREM 9.14 

The Hultman number Sy(n,k) is equal to the number of ways to obtain the 
cycle (12---n(n+1)) € S,41 as a product qr of permutations, where q © Sn41 
is any cycle of lengthn+1, and the permutation r € Sn41 has exactly k cycles, 
that is, c(T'(r)) =k. 


The proof of this theorem is given in Problem Plus 2. 
The following immediate consequence of Theorem 9.14 is more suitable for 
our purposes. 


COROLLARY 9.15 

The Hultman number Sx(n,k) is equal to the number of (n + 1)-cycles q so 
that the product (12---n(n+1))q is a permutation with exactly k cycles, that 
is, C(T'((12---n(n + 1))q) =k. 


PROOF | If (12---n(n+1))q = w, where w has & cycles, and q is an 
(n + 1)-cycle, then multiplying both sides of the last equation by q~! from 
the right, we get the equation 


(12---n(n+1)) =wq"?. 
The claim of the Corollary is now immediate from Theorem 9.14, since q~! is 
a cycle of length n+ 1. 


Example 9.16 

We have seen in Example 9.13 that Sy (3,2) = 5. So by Corollary 9.15, there 
must be five permutations q of length four that consist of a single cycle so 
that (1234)q has exactly two cycles. 
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In order to see that this is the case, note that there are six 4-cycles on 
[4]. One of them, the inverse of (1234), that is, the permutation (4321), is 
clearly ineligible for the role of q since (1234)q = (1)(2)(3)(4) has four cycles. 
The other five 4-cycles are all eligible for the role of gq, since for each of 
them, (1234)q will be the product of two odd permutations, hence an even 
permutation. Hence (1234)q cannot be one 4-cycle, or a 2-cycle and two fixed 
points, and it also cannot be the identity permutation since q is not the inverse 
of (1234). So (1234)q must have two cycles in it for each of the five 4-cycles 
q that are not equal to (4321). 


So finding the average of the numbers c(G(p)) over all n-permutations p is 
equivalent to finding the average of the numbers c(I'((12---n(n+1))q), where 
q is an (n+ 1)-cycle. This is significant, because this translates our problem 
into the language of counting traditional cycles in permutations, which is a 
very well-studied area. 

Let us consider the product s = (12---n)z, where z is a cycle of length n. 
We will monitor the changes in c(I'(s)) as the two factors of s grow in length. 
Let a and b be two entries satisfying z(a) = 6, and let us insert the entry n+1 
into z to get the permutation z’ so that n+1 is inserted “between” the entries 
a and b in the following sense. 


z(t) ifi ¢ {a,n+ 1}, 
z(i) =< n+1ifi=a, and 
bifi=n+l. 
See Figure 9.7 for an illustration. 
The following proposition describes how the insertion of the new entries 


changes the number of cycles of the [-graph, that is, the number of cycles of 
s in the traditional sense. 


PROPOSITION 9.17 
Let a, b, and z’ be defined as above, and let s’ = (12---(n+1))z’. Then the 
following equalities hold. 


1. If 2 < a, and a—1 and 2z(1) are not in the same cycle of s, then 
c([(s’)) = c(T'(s)) — 1 if2<a. 


2. If2 <a, anda—1 and z(1) are in the same cycle of s, then c(T(s’)) = 
c(I(s)) +1. 


3. Ifa=1, then c(T(s’)) = c(T(s)) +1. 
PROOF Let us assume first that a > 2, and that a — 1 is in a cycle C 


of s, and z(1) is in a different cycle C2 of s. Let C; = ((a— 1)b---) and let 
Cy = (2(1)---n). After the insertion of n+1 into z, the obtained permutation 
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FIGURE 9.7 


How 2’ is obtained from z. 


s' = (12---(n+1))z’ sends a—1 ton+1, then n+1 to z(1), then leaves the 
rest of C2 unchanged till its last entry. Then it sends n back to z’(n+ 1) =, 
from where it continues with the rest of C, with no change. So in s’, the 
cycles Cy and C2 are united, the entry n + 1 joins their union, and there is 
no change to the other cycles of s. See Figure 9.8 for an illustration. 

Let us now assume that a > 2, and that a—1 and z(1) are both in the same 
cycle C of s. Then C = ((a— 1)b---nz(1)---). After the insertion of n +1 
into z, the obtained permutation s’ = (12..---(n+1)z’ sends a— 1 ton+1, 
then n +1 to z(1), cutting off the part of C that was between a — 1 and n. 
So C is split into two cycles, the cycle C’ = ((a— 1)(n+ 1)z(1)---) and the 
cycle C” = (b---n). Note that s’(n) = b since z/(n + 1) = b. See Figure 9.9 
for an illustration. 

Finally, if a = 1, then s’(n+1) = (n+1), so the entry n+1 forms a 1-cycle 
of s’, and the rest of the cycles of s do not change. | 


Proposition 9.17 shows that inserting n+1 into a position of z will sometimes 
decrease and sometimes increase the number of cycles of the product s’ = 
(12---(n+1))z. The question is, of course, how many times will an increase 
and how many times will a decrease occur. In light of Proposition 9.17, this 
is the same question as asking how often the entries z(1) and a—1 are in the 
same cycle of s. Furthermore, since z is just an arbitrary n-cycle, and a — 1 
is an arbitrary element of a fixed n-cycle, this is equivalent to the following 
question. 
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n+l 


FIGURE 9.8 
If a— 1 and z(1) are in different cycles of s, those cycles will turn into one. 


QUESTION 9.18 Let TC, denote the set of ordered pairs (x,y) of 
n-permutations that consist of one n-cycle each. Let i and j be two fixed 
elements of the set [n] = {1,2,---,n}. Select an element (x,y) of TC, at 
random. What is the probability that the product xy contains i and j in the 
same cycle? 


Example 9.19 

Let n = 3, then TC;, has four elements, namely ((123), (123)), ((123), (132)), 
((132), (123)), and ((132), (132)). For the pair listed first, we have ry = (132), 
for the pairs listed second and third, we have ry = (1)(2)(3), and for the pair 
listed fourth, we have xy = (123). So for any two distinct elements 7,7 € [3], 
the probability that i and 7 are in the same cycle in the product zy of the 


two permutations in a randomly selected element of TC is 2 — 4. 


Based on this and other numerical evidence, the present author conjectured 
in 2008 that for odd integers n > 3, the answer to Question 9.18 was 1/2. For 
even integers, numerical evidence suggested that the answer to Question 9.18 
was somewhat less than 1/2, so the present author asked what the precise 
answer was. 

Question 9.18 was answered by Richard Stanley [303] in the same year. He 
gave a complicated proof using various high-powered techniques of abstract 
algebra and analysis, such as symmetric functions, character theory, and non- 
elementary integration. Stanley’s result is the following. 
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FIGURE 9.9 
If a— 1 and 2(1) are in the same cycle of s, that cycle will split into two 
cycles. 


THEOREM 9.20 

[3803] Let i and j be two fixed, distinct elements of the set [n], where n > 1. 
Let (x,y) be a randomly selected element of TC,. Let p(n) be the probability 
that i and 7 are in the same cycle of xy. Then 


_ 4 ifn is odd, and 
p(n) ={i3 CECEDI if n is even. 

So p(n) = 1/2 indeed holds for odd integers n > 3. Note that among all 
values that a probability can take, 1/2 is probably the one that is the most 
intriguing in that if a probability of an event is proved to be one half, then 
it is completely natural to ask for a combinatorial, in fact, bijective, proof of 
that fact. Such a proof, which is still not simple, has recently been given in 
[35]. 

It is now not difficult to describe how the average of the Hultman numbers 
grow. Let a, be the average number of cycles (in the traditional sense) in all 
permutations of the set {axy|(z,y) € TC,}. 


LEMMA 9.21 
Letn>1. Then a, = 1, and the numbers ay, grow as follows. 


1. Ifn=2m+ 2, then ayn = an_1 —— 


2. Ifn=2m+1, then Qn = Qn-1 + 4 —- ay 
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PROOF 


iL, 


We apply Proposition 9.17, with n replaced by n — 1, which is an odd 
number. So 2’ is a cycle of length n obtained from a cycle z of length 
n — 1 through the insertion of the maximal element n into one of n — 1 
possible positions. If a 4 1, then a—1 and z(1) are equally likely to be 
in the same cycle or not in the same cycle of s. Therefore, an increase 
of one or a decrease of one in c(G(s)) is equally likely. If, on the other 
hand, a = 1, which occurs in 1/(n—1) of all cases, then c(G(s)) increases 
by one. So 
n—2 1 1 


n— n— 1)= n- =a 
St pag a ee 


an = 


. We again apply Proposition 9.17, with n replaced by n — 1, which is 


now an even number, namely n — 1 = 2m. If a £1, which happens in 
(n —2)/(n—1) of all cases, then the probability of a— 1 and z(1) falling 
into the same cycle of s is 5  C=SIGES) = s- Cmte by Theorem 
9.20. By Proposition 9.17, in these cases c(G(s)) grows by one. Ifa = 1, 
which occurs in 1/(n—1) of all cases, then c(G(s)) always grows by one. 
So 


n-2 (1 1 
Tat (5- Garam) Ot D 
n-2 (1 1 
“ier G+ acne) Ot? 
ar 7 an +2); 


which is equivalent to the statement of the lemma as can be seen after 
routine rearrangements. 


It is now easy to prove an explicit formula for a). 


THEOREM 9.22 
For all positive integers n, the equality 


holds. 


ams | 


[(n 5 0, 


an = Po 
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PROOF This is a direct consequence of Lemma 9.21 and the fact that 


Indeed, if we repeatedly apply Lemma 9.21 to compute a; from a;_, starting 
at dg, then at each step, we will have a new summand equal to 1/(i— 1), and 


in every other step we will also have a summand equal to CES So, assuming 
that n = 2m (the case of odd n is completely analogous), we compute 
1 
Aan = An—-1 17 
tT nl 
tof 1 
TED gees, ead (m—1)m 
n-1 m 
1 1 
i Dae (m—1)m 
t=1 ll 
n—1 
1 1 
= aa =. 
|(n + 1)/2| » i 


Recall the fact, which we proved in several different ways in this book, that 
the average number of cycles of I'(p), taken over all n-permutations, is the 
harmonic number H(n) = 3>i_, +. Theorem 9.22 shows that ay is just a little 
bit more than that. 

Theorem 9.22 now enables us to state and prove an explicit formula for the 
average distance between two randomly selected n-permutations, or equiva- 
lently, the average block interchange distance of an n-permutation from the 
identity permutation. 


THEOREM 9.23 
[65] The average number of block interchanges needed to sort an n-permutation 
1S 

1 nod 
— " — Tin+2)/2] er; 
n 9 * 

PROOF _ By Theorem 9.14 and Theorem 9.22, the average value of c(G(p)) 
of a randomly selected n-permutation p is adn41 = TCEDEN + 4, How- 
ever, Theorem 9.9 states that the number of block interchanges needed to sort 
pis (n+1—-—c(G(p))/2. Taking the average value of this expression over all 


n-permutations proves our claim. 
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9.3.1.1 Variations of the Cycle Graph 


The concept of cycle graphs is a very important one, and as such, it has 
been introduced several times, in slightly different (but essentially equivalent) 
versions. One such version identifies vertices pp = 0 and pry; = n+ 1 so 
that these two vertices become one, and leaves all the other rules unchanged, 
except that i+ 1 and i—1 are now meant modulo n. So the new vertex (let 
us call it 0), now has four edges adjacent to it, namely one edge of each color 
leaving it, and one edge of each color arriving at it, just like any other vertex. 

This “circular” version G’(p) of the cycle graph is sometimes more conve- 
nient to use than the linear version G(p) that we studied in this section. For 
now, Figure 9.10 shows the graph G’(4312) as an example. As the vertices 0 
and n+ 1 have been contracted into one vertex called 0, the vertices of this 
graph can conveniently be arranged in a circle. 


G’(p) 


p=4312 


FIGURE 9.10 
The graph G’(p) for p = 4312. 


In Exercise 6, you are asked to prove that G’(p) has the same number of 
alternating cycles as G(p), no matter what p is. 
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Exercises 


1. 


(—) Prove that if n is odd, then c(G(n---21)) = 2, and if n is even, then 
c(G(n-+-21)) =1. 


. (-) Prove that the identity permutation is the only n-permutation p so 


that c(G(p)) =n +1. 


. Prove that if p = pip2---pn is not the identity permutation, then there 


exists a block interchange that increases c(G(p)) by two that moves a 
block containing the smallest entry p; for which p; 4 i. 


. Let p be an n-permutation. Prove the inequality 


n+1—c(G(p)) 


> 
btd(p) > 5 


. Let us call a cycle of G(p) an odd cycle if the number of black (equiv- 


alently, gray) edges of that cycle is odd. In other words, a cycle is odd 
if it consists of 2m edges, where m is an odd integer. Let coaa(G(p)) 
denote the number of odd cycles of G(p). 


Prove that no block transposition can increase Coda(G(p)) by more than 
two. 


Note that this fact is not a trivial consequence of Lemma 9.11. It could 
possibly happen (but it does not) that a block transposition turns two 
even blocks into four odd blocks. 


. (-) Prove that for any permutation p, the graphs G(p) and G’(p) have 


the same number of alternating cycles. 


. Prove that if in G(p) there is an alternating path from u to v, then u 


and v are part of the same alternating cycle of G(p). 


. (-) Prove that no block transposition on p can change the parity of 


c(G(p)). 


. (-) The breakpoint graph B(p) of the n-permutation p is defined as 


follows. Its vertex set has 2n + 2 elements, with the vertices being 
Lo, Ro, £1, Ri ++: , Ln, Rn. We arrange these vertices so that they form 
a (2n+2)-gon, and Lo, Ro,--- are the labels of the vertices, going coun- 
terclockwise. The edges of B(p) are given as follows. 


(a) For each index i € [0,n], there is a black edge from Rp, to L 
where n + 1 is interpreted as 0, and 


Pi4+1? 


(b) for each index 7 € [0, n], there is a gray edge from L;41 to R;, where 
n +1 is interpreted as 0. 
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FIGURE 9.11 
The breakpoint graph B(4312). 


10. 


11. 
12. 


13. 


See Figure 9.11 for an illustration. 


Prove that for any permutation p, the graphs G(p) and B(p) have the 
same number of alternating cycles. 


(-) 

Let p be a permutation, and let C be a cycle of G(p) with 2k + 1 black 
edges. Then C is called good if it is possible to split C into 2k+1 cycles 
that have one black edge each using k block transpositions so that the 
other cycles of G(p) remain invariant. If C is not good, then C is called 
bad. 


Keep the definition of the graph B(p) from the preceding exercise. Let 
d,—, be the decreasing permutation of length r — 1. Prove that 


(a) if r is even, then B(d,_1) is a bad odd cycle, 

(b) ifr = 4k+1, then B(d,_,) is the union of two bad odd cycles, and 
(c) if r= 4k + 3, then B(d,_1) is the union of two even cycles. 
Sort the permutation 864297531 with five block transpositions. 


Let us say that the permutation p = pip2---p, has a bond in i € (0, n] if 
Pi+1 = pi +1, where po = 0 and ppy1 =n+1. State and prove a slightly 
stronger version of Theorem 9.6 in terms of the number of bonds of p. 


Sort the permutation 963852741 with five block transpositions. 
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15. 
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Let p be an n-permutation, and let us sort p by block interchanges in 
which each block consists of one entry. Find a formula for the number 
of such operations needed to sort p. 


(—) Let p be an n-permutation, and let is(p) be the length of the longest 
increasing subsequence of p. Prove that btd(n) < n— is(p), and show 
an example for an infinite family of permutations for which this upper 
bound is tight. 


The edit distance of a permutation p from the identity is the number 
of block transpositions in which one block is of size 1 needed to sort p. 
Equivalently, this is the number of steps needed to sort p if a step consists 
of choosing one entry of p and placing it somewhere else. Express the 
edit distance of p in terms of a simple parameter of p. 


DT 


Problems Plus 


ile 


2 
3 
4. 
5 


Prove Theorem 9.6. 


. Prove Theorem 9.14. 
. Refine Theorem 9.14. 


Find an explicit formula for Sy(n, 1). 


. Let n > 3, and let i, j, and k be three distinct elements of [n]. Let 


TC), be defined as in Question 9.18. What is the probability that if we 
choose a random element (x,y) € TC;,, then the entries 7, 7, and k are 
in the same cycle of the product ry? 


. Let X,,(p) denote the block transposition distance of the n-permutation 


p from the identity. Prove that there exists a constant c so that 


n Inn 
E(X,) > --—-—e. 


. Let X(p) = c(G(p)). Find Var(X), where the variance is taken on the 


set of all n-permutations. 


. Let ap be the number of derangements p of length n+ 1 that can be 


obtained as (12---(m+1))q, where q is a cycle of length n+ 1. Find a 
formula for the numbers ay. 


. Let a be a partition of the integer n+ 1. Find a formula for the number 


of permutations of length n+1 that can be obtained as (12---(n+1))q, 
where q is a cycle of length n+ 1. 


How 


10. 


11. 


12. 


13. 


14. 


15. 


16. 
17. 


18. 
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In cut-and-paste sorting of permutations, one can move a block of entries 
of a permutation p and insert it somewhere else in p, possibly reversed. 
Show that for any positive integers n, there exists an n-permutation p 
that takes at least |n/2| cut-and-paste moves to sort. 


Prove that for all n-permutations p, the inequality btd(p) < n—coaa (T'(p)) 
holds, where Coaa (I'(p)) is the number of odd cycles of T'(p). 


A prefix transposition is a block transposition that displaces the leftmost 
entry of a permutation. Prove that for infinitely many values of n, there 
exist n-permutations that take |(3n+1)/4] prefix transpositions to sort. 


A prefiz exchange is a block interchange that swaps that first entry 
of a permutation and any other entry of the permutation. The prefix 
exchange distance of the permutation p, denoted by pexc(p) is the min- 
imum number of prefix exchanges needed to sort p. Find an explicit 
formula for pexc(p). 


Let o*) denote the product of k maximal cycles (of length n) chosen 
uniformly at random. Prove that 


1+ (“yet 


P(o) is a cycle) = | 
nm 


(9.4) 


Keeping the notation of the previous Problem Plus, find a formula for 
P(o®) is a cycle). 


Find a formula for P(a?) is a derangement). 


Let n be a fixed positive integer, and let ?;, denote the number of permu- 
tations p of length n whose longest increasing subsequence is of length 
k and whose tableaux risk(p) (defined in Chapter 7) are of the shape 
of a hook. Recall that this means that P(p) and Q(p) are tableaux that 
are unions of their first row and first column. 


Prove that the sequence ¢1, €2,--- ,&n, is log-concave. Explain the con- 
nection of this problem to sorting. 


Let n be a fixed positive integer, and let i; denote the number of invo- 
lutions p of length n whose longest increasing subsequence is of length k 
and for which risk(p) (defined in Chapter 7) are of the shape of a hook. 


Prove that the sequence 71, tl2,--- , in is log-concave. 
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Solutions to Problems Plus 


1. It is proved in [167] by an exhaustive case-by-case analysis that if p 


is not the identity permutation, then it is possible to find two block 
transpositions B and B’ so that one of BB'p, B’pB, and pB’B has at 
least three bonds. The claim is then straightforward by induction. 


. Let p = pip2:-: py be an n-permutation, and let G’(p) be the circular 


version of its cycle graph. Define the permutation 
p:{0,1,---,n} > {0,1,--+ ,n} 
by setting p/(i) = 7 if there is a black edge in G’(p) from i+ 1 to j 
(where n + 1 is identified with 0). Furthermore, set 
p” = (0 Pn Pn-1°*+2 1). 
It is then straightforward to verify that 
pi =p*o(01---n)=(01---n)- px. 


Furthermore, c(I'(p’)) = G’(p). Indeed, each cycle of T'(p’) (that is, each 
cycle of p’ in the traditional sense) corresponds to an alternating cycle 
of G’(p), since if p'(z) = j, then there is a gray edge from 7 to7+1, then 
a black edge from 7+ 1 to j. It is not difficult to show that the map 
g:p—p’ is bijective, completing the proof. This argument is due to 
Jean-Paul Doignon and Anthony Labarre, and can be found in [148]. 


. We have seen in the solution of the previous Problem Plus that there is 


a bijection g from S;, to the set of permutations of length n+1 that can 
be written as a product of (0 1 n) and another cycle of length n + 1 so 
that each cycle of length k in G’(p) is mapped into an alternating cycle 
with & black edges in T'(p’). 


Therefore, if a1, @2,--- ,@n41 1s a sequence of non-negative integers sat- 
isfying a ia; = n+1, then the number of n-permutations p for which 
G'(p) has exactly a; alternating cycles with i black edges for each 7 is 
equal to the number of permutations q of length n+ 1 that can be writ- 
ten as a product of (0 1--- n) and another cycle of length n + 1 such 
that ['(q) has exactly a; cycles of length 2‘. 


. If n is odd, then Sy(n,1) = 0. If n is even, then Sy(n,1) = 5n!. See 


n+2 
[148] for a proof. 


5. Let t(n) denote the requested probability. It is proved in [303] that 


1 1 . . 
e+ iam if n is odd, and 
i(n) = ie a 


3 — Tratymt ifn is even. 
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6. 


10. 


Finding the average value of Coaa(G(p)) is helpful here. By the result 
of Problem Plus 3, knowing that average is equivalent to knowing the 
average of the numbers c(I'(q)), where q is a permutation of length n+1 
that can be obtained as a product (12---(m+1))r, with r being a cycle 
of length n+ 1. 


This average has been computed by Richard Stanley [301], who showed 
that if nm = 2m, then the average of c(I'(q)) taken over the set of n- 
permutations that can be obtained by multiplying (12---n) by an n- 


cycle is 
m 1 2 m-1 1 
om ae (Ste 2k)!(2k — 7) oe 
k=1 : 


k=1 
Stanley also states that a similar formula exists when n is odd. In other 
words, the average number of even cycles of G(p) is just a little bit less 
than half of the average number of all cycles of G(p). The latter was 
computed in Theorem 9.22, where it was found to be close to Inn. Our 
claim can now be proved by a routine computation. 


. It is proved in [238] that 


vax) = (40) + pay) (tee): 


Here H(n) denotes the nth harmonic number. 


. We can assume without loss of generality that ¢ = (1q2---dn41). Then 


p will be a derangement if and only if g¢; 4 gi+1 + 1 module n+ 1 for 
any i. From this observation, a little bit of work yields the recurrence 
relation 


Gn = (n— 2) -an-1 + (n— 1): (2+ Gn_-2 + Gn-3) 


for n > 3, while a9 = 1, a; = 0, and ag = 1. Let A(z) = 30,55 n=. 

Then the above recurrence relation leads to the functional equation 

A(z) = e-*(1 — In(1 — z)), which in turn yields the formula a, = 
n nea n 

(=1)" + pag (-1)*F GG) (n —k - VE 


. There are two ways of computing these numbers, both of which are 


complicated, though sometimes possible. See [148] for a relatively direct 
argument and see [293] for a high-powered argument. 


Let us say that 7 is a parity adjacency for the permutation p = p1p2--: Dn 
if p; and p;41 are of different parity. It can be shown [129] that no cut- 
and-paste move increases the number of parity adjacencies of p by more 
than two. The result now follows if we consider po = 0 and pn4, = n+1, 
and set p to be an n-permutation with only one parity adjacency. 
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. This result is due to Anthony Labarre [239], who proved it as a conse- 
quence of a more complex framework (and not simply showing that in 
each step, Coda (T'(p)) can be increased by one by a block transposition). 


This result is due to Anthony Labarre [240], who showed that all per- 
mutations p for which each cycle in the alternating cycle decomposition 
of G(p) contains an even number of black edges have this property. 


Let f(p) be the number of fixed points of the permutation p = p1p2---Dn.- 
It is proved in [2] that 
0 if p, =1, 
pexc(p) =n —c(I'(p)) — 2f(p) — 
2ifp, £1. 


If n is even, then the claim is obvious, since the product of two n-cycles 
is an even permutation, so it cannot be an n-cycle. If n is odd, then 
the result is equivalent to the fact that the number of ways to write an 
n-cycle as a product of two n-cycles is 2(n — 1)!/(n + 1). This result 
has been discovered many times. See [108] and its references for an 
elementary proof. 


It is proved in [76] that 
N-1 —k+1 
1 N- 
P(o) is a cycle) = s (ayer ‘) 7 
r 


N 
r=0 


It is proved in [76] that 
N-1 T N 
(2) 3 aN. (=) ey) 
P(o isa derangement) =N a (N—7)r! + (N— DY 


The connection to sorting is that we can reverse these permutations (this 
will not change the fact that the shape of their tableaux is a hook), and 
then the longest increasing subsequences will become longest decreasing 
subsequences. The length of those can be used to determine the edit 
distance of a permutation from the identity, as you are asked to show 
in Exercise 16. 


See [77] for a proof of log-concavity. 


This result is proved in [77]. 


10 


Do Not Look Just Yet. Solutions to 
Odd-Numbered Exercises. 


10.1 Solutions for Chapter 1 


1. Note that 
ea ea ica) 
S] S52 — S1 S53 — 82 nm— Sk 
apn 
~ g1'(n— 81)! (82 —81)!"(n— 52)! (n—sx)! 
n! _ n 
81!- (Sg — 81)! +--+: (n—sp)!  \s1,82—81,°-: ,N—Sn/° 


3. If p = pip2:::pPn has k descents, then its complement p° clearly has 
n—1-—k descents. Here p* is the n-permutation defined by (p°); = 
m+ 1— p;. 


5. Look at the sequence {b;}; where b; = a;/aj-1. Then {a;}; is log- 
concave if and only if {6;}; is weakly decreasing, while {a;}; is unimodal 
if and only if once {b;}; gets to a number that is not larger than 1, it 
never grows back above 1. As this second condition on {b;}; is obviously 
weaker than the first, the statement is proved. 


7. We prove the statement by induction on n. Our claim is true for n = 0 


as z'G) =l,andn=lasz=0- Ce) +1. (7). Now let us assume 


that we know the statement for n. Multiply both sides of (1.2) by z to 
get 


grt 2 A(n, k) (“ ae *) Z, (10.1) 


n 
k=0 


Now note that 


a) + (npn (eter), (10.2) 


n n+1 


DOT: 10.1201/9780429274107-10 423 
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Therefore, the right-hand side of (10.1) equation becomes 


Satna NT) + Alyn b+ (FFE) = 


+1 n+1 
“ - —k+1 
een *) atk) + (n= -(7 7" “TY a(n, k—1) 
n+l n+1 
k=0 
aang iy (@PETt*), 
n+1 


where the last step uses the result of Theorem 1.7, with k — 1 playing 
the role of k. Comparing this to the left-hand side of (10.1) proves the 
statement. 


. The crucial observation is that the sum on the right-hand side of the 


equation to be proved has n—k+2 terms. This suggests that we compare 
it to what we get if we apply Corollary 1.19 to compute A(n,n—k+1) = 
A(n,k). (The summation 1.19 would have only n—k+ 1 terms, but an 
additional term in which r = 0 can be added to it without changing its 
value.) That Corollary gives 


n—k+1 
A(n, n—-k+ 1) LD S(n, ryrt( at ) (a1 ere es 
r=1 


n+1l—k-r 
n—k+1 A - 
= | _ -1 n+1—k—r 
Y Sent cares 


which, after the substitution h = n —r, is just what was to be proved. 


This is a classic result due to Frobenius [183]. Compare the coefficients 
of z' on the two sides. On the left-hand side, it is A(n,i). On the 
right-hand side, it is 


n- 


tn i. S(n, k)kI(—1)P-F- 


1 
k=0 
Setting h =n — k, this is precisely the result of Exercise 9. 


For each permutation p € S,, let f(p) be the reverse complement of 
p, that is, the permutation whose ith entry is n + 1 — pryi_;. This 
sets up a bijection from the set of excedances of p onto the set of weak 
excedances of (p°)" that are less than n. Indeed, if i is an excedance of 
p, then i > 1,son+1—i1<n. On the other hand, n is always a weak 
excedance of any permutation, so in particular, of (p°)". This proves 
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15. 


Vhs 


19. 


21. 


23. 


that the number of n-permutations with k —1 excedances is the same as 
that of n-permutations with k weak excedances. Our statement is then 
proved by Theorem 1.36. 


Let T(z) = 0,3, 7n%;. The reader is invited to verify that T’(z) = 
1+T (z)+T?(z)/2 by removing the root of a decreasing non-plane 1-2 tree 
and analyzing the three possible cases. Solving that differential equation 
with the initial condition T(0) = 0, we get T(z) = tanz +secz—1. 


Let us insert the entry n into a permutation of length n — 1 with k—1 
distinct r-falls so that the number of r-descents does not change. To do 
this, we can do one of three things: we insert n between two entries p; 
and pj+1 so that p; > pj41 +7, or we insert n in the last position, or we 
insert n so that it immediately precedes one of the r — 1 entries that are 
larger than n — r. Altogether, this will give us 


(kK-14+1+4+r—-1)A(n—-1,k,r) = (k+r—1)A(n—-1,k,r) 


permutations enumerated by A(n,k,r) in which n is not part of any 
r-falls. 


Now let us insert entry n into a permutation of length n — 1 with k — 2 
distinct r-falls so that the number of r-descents increases by one. We 
know from the previous paragraph that there are k + r — 2 possible 
insertions of n into each permutation enumerated by A(n — 1,k — 1,7) 
that do not increase the number of r-falls. So there aren —k—-—r+2 
that do, providing us with (n—k—r+2)A(n—1,k—1,r) permutations 
enumerated by A(n,k,r) in which n is part of an r-fall. Summing over 
the two cases, we complete the proof. 


No, they are not the same unless r = 1 = @. Indeed, just by looking at 
the definitions of these numbers, we have )>;_, A(n,k,r) = n! for any 
fixed r, but we also have )7;_, Ar(n,k, 0) =n!" for any fixed 2. 


Let us first assume that z is a positive integer. Then the left-hand side 
is just the number of ways to color the elements of [n] so that the color 
of each element is chosen independently from the set [z]. The right- 
hand side is the same, counted by the number of colors actually used. 
Indeed, if m colors are used, then they define a partition of [n] into m 
blocks in S(n,m) ways. Then the colors of the blocks can be chosen in 
2(z —1)---(2 -m+1) = (z)m ways. This proves the claim if z is a 
positive integer. 

Otherwise, note that both sides of 21 are polynomials in z. They agree 
for infinitely many values of z (all positive integers), so they must be 
identical. 


No, that is false. Indeed, let P(z) = 1+2432? and let Q(z) = 1+24+42?. 
Then P(z)Q(z) =1+2z2+ 827+ 723 + 122% fails to be unimodal. 
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(a) Let us assume that p has two alternating runs and starts with an 
ascent. Such permutations increase on the left of n and decrease 
on the right of n. Therefore, we can choose the set of entries that 
precede n in 2”—!— 2 ways, and each such choice will correspond to 
one permutation. As p° has the same number of alternating runs 
as p, this proves G(n, 2) = 2(2"-1 — 2) = 2" — 4, ifn > 2. 

(b) One way to find a formula for G(n, 3) is by using Lemma 1.38 with 
k, = 3, and the above result for G(n,2), to get that for n > 3, we 
have 

G(n,3) = 3G(n — 1,3) + (2” — 8) + 2(n — 3), 


G(n, 3) — 3G(n — 1,3) = 2” + 2n— 14, 


and then by solving this recurrence using the ordinary generating 
function of the sequence G(n, 3). 


To do that, let G(z) = 30,53 G(n,3)z". Then the previous equa- 
tion leads to = 


823 22 1423 

GO) aa a5 qeama ee 1-2’ 
7 2 
eh 8z z 


EON see: 
4z? + 22 1423 
( [ae sas): 


Then we get G(n, 3) as the coefficient of z” on the right-hand side, 


that is, 
G(n,3) = 8(3"-2 — gic IED ge? 7(g-21) 
- se ntl, 
for all n > 3. 


The identity states that the number of all surjections of domain [n] that 
have an even-sized image and the number of surjections of domain [n] 
that have an odd-sized image differ by (—1)”. 


In order to prove this, let f : [n] — [m] be a surjection. Let us first 
assume that it is not the case that f~'(m) = {n}. Let f(n) =i. There 
are two cases. 


(a) If n is the only element of [n] that is mapped into i by f, then 
i #m, and we define f by setting f(x) = f(x) —1 if f(x) > 4, 
and f(x) = f(x) otherwise. Note that f is a surjection from [n] 
into [m — 1], and that n shares its image with at least one other 


element. 
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FIGURE 10.1 
The decreasing binary tree of 263498175. 
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33. 


(b) If n is not the only element of [n] that is mapped into i by f, then 
we define f by setting f(x) = f(x)+1if f(x) >ianda¥n. Note 
that n is the unique element that is mapped to i by f. Also note 
that f is a surjection from [n] into [m— 1]. 


Note that the size of the images of f and f will always differ by one. 
If f-!(m) = {n}, then we set f(n) = m, and recursively proceed with 
this definition with [n — 1] and [m — 1] replacing [n] and [m]. The only 
surjection that will not be matched to another one in this way is the 
identity map of [n], proving our claim. 


We prove the statement by induction on k, the initial case of k = 0 
being trivially true. Now assume the statement is true for k, and prove 
it for k +1. We know from Theorem 1.7 that 


A(n,k+1)-(k+1)A(n—- 1,4 +1) = (n-—k)A(n—-1,k). 


Here the right-hand side satisfies a polynomial recursion by the induc- 
tion hypothesis, so the left-hand side will satisfy that same polynomial 
recursion. Rearranging that recurrence relation, we get a recurrence 
relation for A(n, k). 


For all p € S,, define the decreasing binary tree T(p) as follows. The 
root of T(p) is n, and the left (resp. right) child of n is the largest 
entry of p on the left (resp. right) of n. Then define the rest of the tree 
recursively. See Figure 10.1 for an example. 


It is clear that T is a bijection from S', to the set of all decreasing binary 
trees. Indeed the unique preimage of a decreasing binary tree can be 
read off the tree in order, that is, for each node, read the left subtree 
first, then the node itself, and then the right subtree. 


If a permutation p has exactly k peaks then there are two possibilites. 


(a) When p; < pg. In this case, if pp_1 > pn, then p has 2k alternating 
runs, and if pp_1 < pp, then p has 2k + 1 alternating runs. 
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(b) When p; > pg. In this case, if pp1 > pn, then p has 2k + 1 
alternating runs, and if pp_1 < pn, then p has 2k + 2 alternating 
runs. 


In other words, the set of n-permutations with k peaks consists of the 
entire set of n-permutations with 2k + 1 alternating runs, half of the n- 
permutations with 2k alternating runs, and half of the n-permutations 
with 2k + 2 alternating runs. This proves 


G(n, 2k) + G(n, 2k + 2) 


Peak(n, k) = G(n, 2k +1) + 5 


It is easy to prove, by induction, or otherwise, that the number of de- 
scents of p is equal to the number of right edges of T(p), while the 
number of ascents of p is equal to the number of left edges of T(p). Now 
it is clear that the symmetry of the sequence A(n,k), can be proved by 
the simple bijection that reflects T(p) through a vertical axis. 


Proving unimodality is a more interesting task. Let T’ be a decreasing 
binary tree on n nodes. We define a total order of the nodes of T as 
follows. A node v of T is on level j of T if the distance of v from the 
root of T is 7. Then our total order consists of listing the nodes on the 
highest level of T going from left to right, then the nodes on the second 
highest level left to right, and so on, ending with the root of T. 


Let 7; be the subgraph of T induced by the first 7 vertices in this total 
order. Then 7; is either a tree or a forest with at least two components. 
In the first case, let g(T;) be the reflected image of T; through a vertical 
axis. In the second case, if the components of T; are C, C2,--- ,C;, then 
let g(Ti) = (g(C1), g(C2),---g(C2)). Now prove that if k < |(n—3)/2], 
then 7 can always be chosen so that f(T;) has exactly one more right 
edge than T;. Then an injection from the set of decreasing binary trees 
on n vertices with k right edges into the set of decreasing binary trees on 
n vertices with k + 1 right edges can be defined by finding the smallest 
such i, then replacing T; by g(J;) in T. 


(a) The first proof of this fact is due to Gabor Hetyei and Ethan Reiner 

[215], who used exponential generating functions and partial dif 
ferential equations to get this result. The proof we present is com- 
binatorial. 
For any 7 < n—1, either p; is an ancestor of pj41, or pj+1 is an 
ancestor of p;, or else p; and p;4; would have a common ancestor, 
which would put an index between i and i+ 1. In particular, T7” 
cannot have both p; and p;+; as leaves. This makes the following 
definition meaningful. Let i € [n—2]. Then the ith local extremum 
of a permutation p is the entry that is closest to the root of the 
minmax tree of p among p;, pi41, and pj+2. Denote this entry by 
e;. Note that e; always exists. 
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Let us assume without loss of generality that the entry 1 of p pre- 
cedes the entry n of p. We can do that as the minmax tree of p° is 
isomorphic to 77”. 

First we are going to prove that p, is a leaf in n!/3 minmax trees. 
The simplest scenario is when the entry 1 is among the leftmost 
three entries, so in particular, e; = 1. This gives rise to three 
subcases: 


e If py =1, then p, is the root of the minmax tree. 

e If pp =1, then pg is the root, and py is a leaf. 

e If ps = 1, then ps3 is the root, its left subtree has p; and p2 as 
nodes, and among these, by definition, pg is a leaf, and p; is 
not. 


Clearly, these cases are equally likely to occur, so p; will be a leaf 
with probability 1/3 in this case. 

Now let us assume that the entry 1 of the permutation is not 
among the first three elements. This entry is the root of 7)”, 
and its left subtree has at least three nodes. Let these nodes be 
by < bg <--+ < by. Repeat the previous argument for this subtree, 
with b; playing the role of 1; now if b; is among the leftmost three 
elements of the permutation, p, is a leaf with probability 1/3. Iter- 
ate this algorithm. It will eventually stop because we either get a 
left subtree of size three, or a subtree whose minimal entry is among 
the first three ones. This proves that there are n!/3 minmax trees 
on [n] in which py is a leaf. 

The proof for general p; is similar. Assume again that 1 is on the 
left of n in p, and let 7 € [n — 2]. If 1 € {p;, pi41, Di42}, then there 
are three possibilities. 


e If p; =1, then p; is the root of the minmax tree. 


e If pj41 = 1, then p;+1 is the root, so p; is the rightmost element 
of its left subtree, and as such, it is necessarily a leaf. 


e If pi4o = 1, then p;+2 is the root, p; is the next-to-last element 
of the root’s left subtree, and as such, it is always an internal 
node (having the leaf p;+1 for its only child). 


Again each of these subcases occurs with probability 1/3. 

If 1 ¢ {p;, pi+1, pit+2}, then we can proceed as above. That is, look 
for the entry 1 of p, then only consider the subtree that contains 
the positions 7, i+ 1, andi+ 2. If 1 is not in any of these po- 
sitions, then all three of them are in the same subtree. Iterating 
this algorithm, we eventually reach a subtree where we can apply 
the above method. The structures of the other subtrees do not 
influence whether p, is a leaf or not, so p; is a leaf with probability 
1/3. This completes the proof. 
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(b) Clearly, p, is always a leaf because it cannot be the leftmost in 
any comparison, thus it cannot have descendants. Similarly, p,, is 
always the child of pyn_1, thus py_ 1 is never a leaf. 


Any lattice path included in our sum must end either in a horizontal 
step having label & or a vertical step having label n —k +1. Then use 
induction and Theorem 1.7. 


Similar to the solution of Exercise 39, just this time we have to use the 
recurrence proved in Exercise 17. 


Note that when we insert two ns into an element of Q,_1, we must insert 
them into consecutive positions. All the claims now follow by analyzing 
how many times this insertion increases the value of the statistic at 
hand. 


Reversing the order of the last two entries of a permutation p changes 
as(p) by plus or minus one, and so it changes its parity. 


a 


10.2 Solutions for Chapter 2 


1. 


There are two known solutions of these results. One [321] is by a com- 
binatorial involution, and the other [142] is by a generating function 
argument. These solutions generalize in different directions. 


As we discussed, b(n, 3) is equal to the coefficient of x? in the polynomial 
In(z) = (14+ 2)\(1+2+27)---(l+2+---+2"7'); in other words, b(n, 3) 
is the number of weak compositions of 3 into n — 1 parts, so that the 
first part is at most 1, and the second part is at most 2. 


There are CS) weak compositions of 3 into n—1 parts. One has second 
part 3, one has first part 3, and n—2 of them have first part 2. Therefore, 
b(n, 3) = ("F") — 1. 


The number of pairs of entries (x, y) so that « > y+1 is ($) -—(n-1) = 
Cok Each such pair appears in reverse order in half of all permuta- 
tions, so the answer is n!("5*) /2. 


The number of such trees with zero inversions is n!. Figure 10.2 shows 
the six trees with root 0, non-root vertex set [3], and no inversions. To 
prove this statement bijectively, let p be an n-permutation, and define 
H(p) to be the tree on vertex set [n] U0 defined as follows. If i is an 
entry of p, then the unique parent of 7 in H(p) is the vertex j, where j 
is the closest entry in p that is on the left of 7, and that is smaller than 
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The six trees with no inversions. 


t. If there is no such entry 7, then the parent of 2 is 0. The reader is 
invited to verify that H is indeed a bijection. 


9. The following proof was given by David Bressoud and Doron Zeilberger 
[99]. Let A = (Aq, Aa,-+: , Az) be a partition of n — a(j), for some even 
j. Define o(A) by 


(t+ 37 —1,A1-1,A2 —1,---A, — 1), iff +37 > Ad, 


oA) = 
(Ag +1,---Ate £1,1,1,---,1) if t +37 < Ad. 


If the second rule is used, then there are 41 — 37 — t— 1 parts equal to 
1 added at the end. 


Note that the first rule sends \ into a partition of n — a(j) +37 =n — 
a(j—1), and the second rule sends X into a partition of n—a(j)—3j-2 = 
n—a(j +1). So in both cases, ¢ maps from U; even Par(n — a(j)) into 
U; oaaPar(n — a(j)). Finally, note that applying ¢ twice is the identity, 
therefore @ has an inverse, and must be a bijection. 


Another solution was given by Adriano Garsia and Steve Milne [188]. 


11. Yes. We prove the statement by induction on k, the initial case of k = 0 
being trivial. Suppose we know that the statement is true for k — 1. 
Lemma 2.5 tells us that b(n + 1,k) — b(n, k) = b(n + 1,k-1)ifk <n. 
From this, it is easy to see that b(n,k) is also p-recursive. Finally, we 
mention that the constraint k < n is not a reason for concern. Indeed, 
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it is obvious that if f and g are polynomially recursive functions, then 
so is f +g. On the other hand, for any fixed k, the difference of the 
functions b(n + 1,4 — 1) and b(n + 1,k) — b(n, k) is nonzero for a finite 
number of values of n only, and is therefore certainly p-recursive. 


There are b(n, k) such n-tuples. Indeed, let p = pip2-+-+ pn be a permu- 
tation, and define b(p); to be the number of indices 7 < i so that p; < pj. 
Then clearly 0 < b(p); < i—1, and the map b: S,, > B, defined by 
b(pip2---Pn) = (b(p)1, b(p)2,--- , b(p)n) is a bijection because the only 
preimage of (6, be,--- ,b,) can be built up from left to right. The value 
of b; reveals the relative size of p; among the first 7 entries of p. Finally, 
note that S>i_, b(p); = i(p) as b(p); is the number of inversions whose 
first element is p;. Therefore, the set of permutations with k inversions 
is mapped to the subset of B, in which $7", b; =k. 


We note that (6(p)1, b(p)2,--- ,b(p)n), or a trivial transformation thereof, 
is often called the inversion table of p. 


In an n-permutation with k inversions, the entry n can be part of 7 
inversions, with 0 <i<k, so 


k 


b(n, k) = 5° b(n —1,k - 2). 


i=0 


Such a permutation is either alternating, or reverse alternating, but 
in both cases, it has k descents and k& ascents. Therefore, it has to 
have at least k inversions and at least k noninversions. These are both 
attainable, with the permutations 13254 --- (2k +1)(2k) and its reverse. 


The main idea of this solution can be found in [174]. Just as in the 
solution of Exercise 13, we are bijectively encoding a permutation by an 
n-tuple (di, d2,--- ,dn) of nonnegative integers satisfying d, < k — 1. 
This will prove that den is Mahonian. We define the d; as follows. Let 
P= Pip2:**Pn be an n-permutation. For fixed k, let 


l{l<k so that pp < py < k}| if pe <k, 
di, = di (p) = 


{1 <k so that py < k}|+|{l <k so that pp < pi} 
if pp > k. 


It is easy to check that )>;"_, di(p) = den(p). All we have to show is 
that p can be recovered from its Denert-table, that is, from the n-tuple 
(di(p), do(p),--+ ,dn(p)). Indeed, pp = n —d,. Now assume that the 
last n — k elements of p have already been recovered. Then recover p, 
as follows. Look at the list k,k—1,---,1,n,n—1,---,k4+1, and delete 
all the entries that have already been assigned to a position in p. Then 
px has to be chosen so that there are exactly d;(p) entries on its left. 
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Let us multiply both sides of the identity by [(m — k)]![k]! to get 
[n]! = [n — 1]![k]q”~* + [n — 1]![n — kl. 
Now let us divide both sides by [n — 1]! to obtain 
in] = [klg-* + [n — K], 
which follows directly from the definition of [ml]. 


This can be proved by repeated applications of the result of Exercise 21, 
but we prefer a combinatorial argument. The left-hand side provides 
the generating function for all k-subsets of [m] according to their subset 
sums. 


A typical term of the right-hand side is of the form q™~*—J+! faz 
where j € [m —k]. We claim that this term will provide the generating 
function for those k-subsets of [m] whose largest element is equal to 
m—j+41. Indeed, the rest of such a subset is a (k — 1)-subset of the 
set [m —j]. The term g™—J-**! corrects the shift in the definitions of 
a; and c;, as seen in the proof of Theorem 2.27. 


We claim that there is a bijection between the Ferrers shapes consisting 
of n boxes that fit within an i x k rectangle and k-subsets of [¢ + &] 
whose sum of elements is n + eae Indeed, let F' be the Ferrers shape 
of a partition of n into at most 7 parts of size at most k. Let the row 
lengths of F' be (f1, fo,--: , fe) im nonincreasing order, where some f; 
may be equal to 0. Then {fi +k, fo+k—1,---, fe +1} is a k-element 
subset of [i + k], and the sum of its elements is n + (“'). As this map 
is obviously a bijection, our statement is proved by Theorem 2.27. 


See [333] for a bijective proof of this fact. 


If n is odd, then we recall that Fal is the number of k-dimensional sub- 
spaces of an n-dimensional vector space over a qg-element field. Match- 
ing each such subspace with its orthogonal complement, we get that 
De=o(—1)*[] = 0. 


If n = 2m is even, then we claim that 


n ii _ 

Det] =G-9G-@)-- 0-9"). 

k=0 
A computational proof of this fact can be found in [13]. Recently, a 
more combinatorial proof was given in [139]. 


Consider the set of insertions that increase the number of descents of p 
and the insertions that increase the number of ascents of p separately. 
Show that both sets of insertions create a set of permutations with 
distint major indices, and that both sets of these major indices will 
form an interval. 
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Let G be a graph on five vertices, so that vertices A, B, C, and D form 
a complete graph except that BC is not an edge, and let BE be the last 
edge of G. 


Let G be the inversion graph of the n-permutation p. Let P, be a poset 
on vertex set [n] so that i <p j if ¢ < 7 as integers and 7 precedes j in 
p. Then the comparability graph of P, is precisely G. So all inversion 
graphs are comparability graphs. 


For a comparability graph that is not an inversion graph, consider the 
poset Q with elements {Ai,--- , Az, Bi,--- , Ba}, in which A; < B; ifiF 
j, and there are no other comparable pairs. Let G be the comparability 
graph of Q. Then G is a connected graph in which each vertex has 
degree three. 


When we try to construct a permutation p = p,---pg whose inversion 
graph is G, we quickly notice that the only way in which p, and pg can 
each be of three inversions is by py = 4 and pg = 5. Similarly, py = 1 
and ps = 8 must hold. After this, it is straightforward to verify that 
the only candidate for p that satisfies the requirement that each p; is 
part of three inversions is p = 43218765. However, the inversion graph 
of that permutation is the disjoint union of two complete graphs on four 
vertices, hence it is not isomorphic to G. 


(MR 


10.3. Solutions for Chapter 3 


di, 


Interchanging the first two elements of a permutation p either increases 
or decreases the number of inversions of p. In either case, it changes the 
parity of p. So exactly half of all permutations are odd, and half are 
even. 


Let (a,b) be an inversion of p. That means that looking at p as a 
function from [n] to [n], we have p(i) = a and p(j) = b, with a > b and 
i <j. Then, by the definition of the inverse, we also have p~!(a) = i 
and p-!(b) = j, proving that (j,7) is an inversion of p~!. This sets up 
a bijection between the inversions of p and p—!. 

Let f(p) = 1 if p is even, and let f(p) = —1 if p is odd. In other words, 
f(p) = det Ay. Note that this map f is often called sign. 


We claim that f must be the identity permutation. Let us assume that 
this is not the case, then f contains a k-cycle (fi fo--- fx), with k > 1. 
Let us first assume that k > 2. Let g = (fife). Then f-g 4 g-f. Indeed, 
IF(fi)) = g(f2) = fi, while fg(fi) = f(f2) = fs. If, on the other hand, 
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f has no cycles longer than two (that is, k = 2), then let g = (ffs), 
where 3 is any element outside the cycle (f; fo). Then again f-g 4 g-f. 
Indeed, g(f(f1)) = 9(f2) = fs, whereas f(g(fi)) = f(fi) = fe. 


. Such a permutation has a unique longest cycle C’, of length k. We have 


(:) (k — 1)! choices for this cycle, then (n — k)! choices for the rest of the 
permutation. Therefore, our total number of choices is 


As each permutation is a product of its cycles, it suffices to prove our 
statement for permutations that consist of one cycle only. This is not 
difficult to do by induction as 


(a1a2q-+- ax) = (a1: ++ ae—1)(ae-14k). 


We claim that n = 3 is the only such value. Indeed, Sj has two 
conjugacy classes of size 1. If n > 5, then S, has two conjugacy 
classes consisting of mT D elements. These are those that belong to 
types (2,1,0,0,--- ,0,1,0,0,0) and (0,0,0,1,--- ,0,1,0,0,0). For n = 
4, these classes both have six elements. 


We point out that much more is known. F. Markel [251] conjectured in 
1973 that S3 was the only finite solvable group that had no conjugacy 
classes of the same size. This conjecture stayed open for twenty years, 
and was then proved independently in [228] and [340]. However, it is 
not known at this time whether the condition that the group be solvable 
can be dropped. 


Multiplying both sides by fr from the right, we get that our formula is 
equivalent to 
frCr = (123---n) fr. 


Compute where each sides takes the node k. For the left-hand side, we 
get 


frCr(k) = Cr(fr(k)) = Cr(Cp*(1) = Cp(1) = fr(k + 1). 
For the right-hand side, we get 


(123---n) fr(k) = fr((123---n)(k)) = fr(k + 1). 


Clearly, c(n, 1) = (n—1)! as such permutations have type (0,0,--- ,0,1). 


We have c(n,n — 1) = ) as all such permutations must have type 


(n — 2,1,0,--- ,0), and we have i) possibilities for the single 2-cycle. 
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We claim that 


n! 1 


! rylrg!- s.r? 
retreat +re=n sae e 


where the r; are positive integers. Indeed, order the elements of [n] in 
one of n! ways, then insert a bar after the first r, elements, then the next 
rp elements, and so on. This provides a partition counted by S(n,k). 
However, each such partition will be obtained in k!- ry!ro!---r,! ways, 
as the order of the elements within each block does not matter, and the 
order of the blocks does not matter. 


This approach to Stirling numbers was studied in [222]. Both polyno- 
mials have degree 2k and leading coefficient oe: This is straight- 
forward to prove by induction, using the recurrence relations given in 


Exercise 8 of Chapter 1 and Lemma 3.19. 


We will show that the right-hand side also counts all permutations of 
length n+ 1 with k +1 cycles. Let the entry n+ 1 be part of an 
(n —m + 1)-cycle C. Then we have ade ways to choose the n — m 
remaining elements of C, and we have (n — m)! ways to choose C’ on 
these elements. Then, we have c(n,m) ways to choose the rest of the 
permutation. Summing over m, we get the identity to be proved. 


We claim that 


c3(n, k) = (n — 1)e3(n — 1,k) + (n — 1)(n — 2)e3(n — 3,k — 1), 


with c3(0,0) = 1. Indeed, in a permutation enumerated by c3(n,k), 
the entry n is either in a 3-cycle, or in a larger cycle. There are n — 1 
permitted ways to insert n into a gap position of a permutation counted 
by c3(n — 1,&) as the last gap position is forbidden (it would create a 
1-cycle). The permutations obtained this way will contain n in a cycle 
longer than three. 


Otherwise, there are ene ways to choose two entries that can share a 


3-cycle with n, then there are two possible 3-cycles involving n and the 
two chosen elements. 


Let p(k,n) be the probability that we draw k white balls in n trials. 
That event can occur in two different ways, either we get the k — 1 
white balls during the first n — 1 trials, or we get only k white balls 
during the first n — 1 trials, and the last one during the last trial. This 
leads to the recurrence relation 


k = 
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Indeed, if the first a trials resulted in drawing b white balls, then there 
are m — b white balls, and 6 black balls in the box. 


This triangular recurrence is somewhat similar to that of the Stirling 
numbers of the second kind. To grasp this connection better, set 


(m)i 


p(k, n) = d(n, k) 


Then the numbers d(n,k) satisfy the same recurrence as the numbers 
S(n,k), and fulfill the same initial conditions. Therefore, d(n,k) = 
S(n,k), and thus p(k,n) = S(n, k) 2. 

This is a simple application of Corollary 3.48. We take a partition of [n] 
into k parts, but we do not put any structure on any of the blocks. The 
only requirement is that the blocks are not empty. Therefore, F;(u) = 
e“ —1 for alli € [k]. Finally, unlike in Corollary 3.48, our partitions are 
unordered, so their number is only 1/k! times what it would be if they 
were ordered. This yields 


By repeated applications of Proposition 3.12, we have 


p! 


e(p, k) = @!a!---apl1%12%--. par 


ai, +2a2+::-+pap=p 


As p is a prime, no positive integer smaller than p is divisible by p. 
Therefore, the denominator of no summand on the right-hand side is 
divisible by p. Indeed, we must always have ap = 0, otherwise we would 
have a permutation with one cycle only, and that is not allowed. As p! 
is divisible by p, the right-hand side is the sum of several integers, each 
of which is divisible by p. 


Let p > n > 1, and let the matrices s and S be defined as in Theorem 
3.30. As s-S = J, the nondiagonal entries of this matrix are zero, so 
for p # k, we have 


0= 5° s(p, 4)S(é,k) = > s(p,i)S(i, k). 


P 
t=1 i=k 


We know from Exercises 31 and 32 that s(p,7) is divisible by p unless 
«=1ori=p. Therefore, the only summand of the far-right-hand side 
in which the s(p,2) term is not divisible by p is s(p,p)S(p, k) = S(p,k). 
As the right-hand side is divisible by p, so too must be S(p, k). 
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As n> 1, we can divide (3.3) by z to get 


n 


(24+1)---(2+n-1) =) e(n,k)2*?. 


k=l 
Now equate the coefficients of z*~! on the two sides. 


Let A; be the set of n-permutations in which 7 is a fixed point. Then 
|A;, N---M Aj,| = (n—k)!, and the result follows by the Principle of 
Inclusion—Exclusion. 


One of these sets is always one larger than the other. For odd n, the 
set of n-permutations with exactly one fixed point is larger. For even n, 
the set of derangements of length n is larger. 


To see this, let G(n) be the number of n-permutations with exactly one 
fixed point, and note that G(n) = nD,_, holds for all n > 3. Indeed, 
to find a permutation counted by G(n), first choose its only fixed point, 
then choose a derangement on the remaining n—1 entries. On the other 
hand, it follows from Corollary 3.57 that D, = nDn—1+(—1)”". This 
proves our claim. 


The notion of desarrangements, and the proofs below, are due to Jacques 
Désarmenien [141]. 


(a) Let p be a desarrangement of length n. Delete its last entry, and 
relabel the remaining entries accordingly. The obtained permuta- 
tion p’ is always a desarrangement of length n — 1, except when 
p = n(n —1)---21 and n is even. Conversely, each p’ is ob- 
tained from n different n-desarrangements p this way, except for 
p! = (n—1)(n—2)---+21 when n is odd. The latter is only obtained 
from n — 1 desarrangements of length n as 1 could not be the last 
entry. 


(b) A routine computation shows that the numbers D,, as given by 
Corollary 3.57 satisfy the same recurrence relation as the numbers 
J(n) have been shown to satisfy in part [(a)] of this exercise. As 
J(1) = D, = 0, the proof follows. 


The entry 1 of a derangement p of length n can be part of a 2-cycle or 
a larger cycle. There are (n — 1) other elements it can form a 2-cycle 
with, and then the remaining n — 2 elements can form a derangement 
in Dn—2 ways. On the other hand, if 1 is not to be in a 2-cycle, then we 
can just insert it in any gap position of any (n — 1)-derangement (taken 
on the set {2,3,4,--- ,n}) except the one that would put this entry into 
its own l-cycle. This provides (n — 1)D,—-1 additional derangements. 
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We will repeatedly use the triangular recurrence relation c(m,k) = 
c(m—1,k-1)+(m—1)c(m—1,k). Subtract n+k—1 times the next-to- 
last row from the last row. Then the 7th element of the last row becomes 
c(n+k,i)-—(n+k—-1)c(n+k-1,1) =c(n+k—1,i-—1). Now subtract 
n+k—2 times row (k—2) from row k—1. This results in a row k—1 whose 
ith element is c(n+k—1,1)—(n+k—2)c(n+k—2,i) = c(n+k—2,i-1). 
Continue in this way for all rows. We get the matrix 


c(n + 1,1) c(n+1,2) -:- c(n + 1,k) 
L LAs), x (n+ 1k = 1) 


c(n+k—1,0) cn+k—-1,1)--- elie Ho k—1) 


that is, a matrix whose first row is identical to that of C,,, but in which 
the jth element of row 7 is cin +i—1,j — 1), fori > 1. Expanding 
this matrix with respect to the first column and using induction (the 
(n — 1) x (n — 1) minor in the lower right corner is C,_1), we get our 
claim. 


List the elements of [2n] in any of (2n)! ways, then insert bars after 
every two elements. This will result in a fixed point-free involution. On 
the other hand, we obtain each such involution n! - 2” times as we can 
change the order of the elements within each pair, and we can change 
the order of the pairs without changing the involution itself. 


We use the Compositional formula, with g(m) = m! and f(k) = (k—1)! 
for k > 1. This yields F(z) = —In(1 — z), G(z) = 1/(1— z), and so 


1 
~ 14+In(1 - 2)’ 


In such a permutation, each cycle length must be either one or three. 
Therefore, by Theorem 3.53, we get 


Gx(2) =e (2+ 5) = Sy 


i=0 j>07 
n (n/3] 
ar. z n: 
=o = 33 9!(n — 39)!, 


showing that g,(C) = se HE CEA The reader is urged to find a 
combinatorial explanation for this formula. 
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By repeated applications of the method seen in Example 3.64, we get 


Cie Seonies (=) ace (=) a (=) = 


II 

co) 

ta 

os 
———N 
Mae 
ho| & 
|p 
NY 
} 

ie) 

a 
=a 
oo 
nN 

Ld 

| 
Km BR 
Scat” 


(This solution is due to Dennis White [53].) Let p € ODD(2m). Denote 
by C1, C2,--- ,Cox the cycles of p in canonical order. We construct a 
bijection ® : ODD(2m) > EVEN(2m) as follows. For alli, 1<i<k, 
take the last element of C2;-1, and put it at the end of Co; to get ®(p). 
For example, if p = (4)(513)(726)(8), then ®(p) = (5134)(72)(86). Note 
that if Co;-1 is a l-cycle, it disappears, and that the canonical cycle 
structure of p is maintained. 


To see that ® is a bijection, it suffices to show that for all o € S,, we 
can recover the only permutation p € ODD(2m) for which ®(p) = o. 
While recovering p, we must keep in mind that it might have more than 
h cycles, because some of its 1-cycles might have been absorbed by the 
cycles immediately after them. If the last value in cp, is larger than 
the first value in cy_1, then create a 1-cycle with this value, placing 
it in front of c, and repeat the whole procedure using cp_2 and cp_}. 
Otherwise, move this value from cp, to the end of cp_; and repeat the 
whole procedure using c,-3 and cp_2. If at any point only one cycle 
remains, create a l-cycle with the last value in that cycle. It is then 
straightforward to check that the permutation p obtained this way fulfills 
®(p) =o. It also follows from the simple structure of ® that at no point 
of the recovering procedure could we have done anything else. 


By Theorem 3.53, the exponential generating function of these permu- 
tations is given by 


Gc(x) = exp S- — 


nZékr 


The argument of exp on the right-hand side has to be computed a little 
bit differently from the special case of k = 2, which was covered in 
Corollary 3.59. 


We have 


yn gn gkn 
ye-ye-ye 


nZékr n>1 n 
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Therefore, 
1 1—2k 1/k 
Gc(z) = exp (inca a 7 nd a) = a 
Now note that 1— z* = (1— z)(l+z2+2?+---+2*71), from which we 
get that 
Go(z) = (Lt 2427 4---42' YY — 24) G-H/e, (10.3) 


Let us assume for brevity that n = mk. Applying the binomial theorem, 
we get 


1 1-k 1-2k 1-—(m-1)k 


= sai] m km eats . 
pa Nake eer Ik Ik 
3 pel k&-1 2k-1 (m—1)k—-1 
a mk k k , 
m>0 


where n = mk. 


So the coefficient of ee = a7 rE @ (omer i Lima eae 


(kn)!(k — 1)(2k —1)---((m—1)k—1) 
mikm 


=1-+.+(k-—2)(k=1)?(k+1):-- (2h -—1)?(Qk + 1)-++(n — 1). 


n 


It follows immediately from (10.3) that a, is also the coefficient of a,,/n! 
in Gc(x), in other words, an = gc(n). Finally, if n is not divisible by 
k, then the only difference is that the long product in our last formula 
has last term n, not n — 1. 


Take a pair (7,k) € ODD(2m+1) x [2m+ 1], and insert 2m-+ 2 into the 
kth gap position. Note that this implies 2m+2 cannot create a singleton 
cycle as it cannot go into the last gap position. Take away the cycle C’ 
containing 2m + 2, and run ® of the solution of Exercise 55 through 
the remaining cycles. Then, together with C’, we have a permutation in 
EVEN(2m). Run it through ®~! to get r(7,k) € ODD(2m + 2). 


We claim that 7 is indeed a bijection. To get the unique preimage of 
mw’ € ODD(2m + 2) under 7, run 7’ first through ®. This way 2m + 2 
gets into the first position of an even cycle, and therefore it indicates a 
gap position, which is not the last one, and thus we recover k. Remove 
2m+2, leave its cycle intact, and run the remaining even cycles through 
&~! to get r71(r’). 
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61. This argument is due to Dennis White [53]. In this solution, we are 


using the bijection UV : ODD(2n) x [2n + 1] > ODD(2n + 1) of the 
solution of Exercise 58. If the reader has not solved that exercise yet, he 
is urged to do so now. As a hint, he may use the solution of Exercise 59. 
The maps ® used in the solution of Exercise 58 and 59 are very similar. 


Let SQ(m) denote the set of all m-permutations with a square root. 
We are going to construct a bijection « from SQ(2n) x [2n + 1] onto 
SQ(2n+1). As the growth of |SQ(n)| is equal to that of |ODD(n)| 
when passing from 2n to 2n + 1, we try to integrate the bijection WU : 
ODD(2n) x [2n + 1] — ODD(2n + 1) into k, by “stretching” the odd 
cycles part of our permutations. We proceed as follows. 


Let (7,k) € SQ(2n) x [2n+ 1]. Take a, and break it into even cycles 
part and odd cycles part, or, for brevity, odd part and even part. Let 
k mark a gap position in a. If this gap position is in the odd part, or 
at the end of z, then interpret the gap position as a gap position for 
the odd part only, and simply run the odd part and this gap position 
through W to get «(m), together with the unchanged even parts. Note 
that 2n + 1 will appear in an odd cycle when we are done. 


If the gap position marked by & is in one of the even cycles, say c, we 
can think of it as marking the member of c immediately following it, say 
xz. Replace x by 2n+ 1 in c. To keep the information encoded by x, we 
interpret x as a gap position in the odd part of 7. Indeed, if x is larger 
than exactly 7 — 1 entries in the odd part, then let us mark the 7th gap 
position in the odd cycles part. So now we are in a situation like in the 
previous case, that is, the gap position is in the odd part. 


Run the odd part and this gap position through V. Instead of inserting 
2n + 1 to the marked position, however, insert temporarily a symbol 
B, to denote a number larger than all entries in the odd part. Then 
decrease all entries in the odd part that are larger than x (including B) 
by one notch. The obtained odd cycles and the unchanged even cycles 
(except for the mentioned change in c) give us K(). Note that 2n +1 
will be in an even cycle when we are done. 


Now we show that « is indeed a bijection. First, it is clear that « maps 
into SQ(2n + 1). Indeed, (7) and «(7) have the same number of cycles 
of each even length, so 7 € SQ(2n) implies «(7) € SQ(2n + 1). 

For example, let 7 = (31)(65)(742)(8) and let & = 3. Then & marks 
the entry 6. So we replace 6 by 9, get the new even part (31)(95), and 
turn to the odd part, (742)(8). In it, the entry 6 marks the third gap 
position as it is larger than two entries, 2 and 4. So we have to apply UV 
to ((742)(8),3). When we do that, first we get ®((742)(8)) = (74)(82), 
then we insert B into the third gap position to get (74)(B82). Now we 
decrease the entries larger than 6 by one notch: B to 8, 8 to 7, 7 to 6, to 
get (64)(872). Finally, we apply &~! to (64) and complete the odd part 
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of «(7), that is (4)(6)(872). With the previously obtained even part, 
this yields that «(a) = (31)(4)(6)(872)(95). 

To get the reverse of «, take a permutation 7’ € SQ(2n + 1), and locate 
2n +1. If it is in an odd cycle, then run the odd cycles through U~?. 
This will yield an odd part one shorter, and an element of [2n + 1]. 
Putting this together with the unchanged even part, we get K~1(z’). 


If 2n + 1 is in an even cycle, then run the odd part through U~!. This 
will specify a gap position in the odd part, and so we recover the entry 
x. Increase entries larger than x by one notch in the odd part. To get 
the even part, put « back to the place of 27 +1. The gap position 
immediately preceding 2n + 1 is our k in K~1(7’). 


The trials of Exercise 6 of Chapter 1 are not independent in the sense 
that Lévy’s theorem requires them to be. That is, for Lévy’s theorem to 
be applicable, we have to define what a success is in these trials. Clearly, 
a success has to be defined as the event that the ball currently placed 
goes into a box that was previously empty (this is how the numbers 
of empty boxes will equal the Eulerian numbers). However, with that 
definition, the trials are not independent as the probability of success 
on trial 7 does depend on the number of successes on the previous trials 
if i > 3. Therefore, Lévy’s theorem does not apply, except when n < 2. 


Take the derivative of both sides, then set z = 1. We get the identity 


n n—1 nl 
d elnk) = » Toa 


Now divide both sides by n! and notice that we obtain the equality that 
was to be proved. 


Use the Exponential formula with f(1) = u and f(n) = (n — 1)! for 
n>1. Then 


1 
Feu) =uz+ = =(u-1)e+In( i; 
—2Zz 


n>2 


Therefore, by the Exponential formula, we have 


-—1 
H(z,u) = exp F(z, u) = SvuSte Ls 
l-z 
Let g and S be as described in the exercise. Let M be the largest element 
of the set F' consisting of all fixed points of ¢g and all elements of [n] that 
are not in S. 


Now let f(q,S) be obtained from (q,) by simply switching the status 
of M. That is, if M is a fixed point of g, then move M outside S. If M 
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is outside S, move it to S and let M be a fixed point of the involution 
taken on S. 


This sets up an involution on the set of all allowed pairs (q,S) that 
changes the parity of the length 7 of g. This involution is not defined if 
and only if M does not exist because the set F’ is empty. That happens 
when 7 = n and q has no fixed points. This completes the proof. 


It suffices to show that the map has an inverse. Let T be a non-plane 
tree on vertex set {0,1,--- ,n} that satisfies the conditions on the labels. 
Then 0 must be the label of the root of T. We can reconstruct the unique 
inverse image of 7’ as follows. The children v, > v2 >--: > vz of 0 are 
the left-to-right minima of this inverse image p. As the left-to-right 
minima of any permutation are in decreasing order, the order of these 
left-to-right minima is given. 

We then recover the segment of p that starts at v; and ends right before 
vi41 by inductively applying this procedure to the subtree of T that is 
rooted at vj. 


Let us count all such trees with a neighbor of a leaf marked. Let H(z) 
be the exponential generating function of such structures. Removing 
the root of such a tree, we get a collection of trees, one of which has a 
neighbor of a leaf marked, while other trees are simply increasing trees, 
and their set is in bijection with permutations. The only time when 
this does not happen is when the marked vertex was the root itself, and 
it has been the neighbor of a leaf. So in this case, after removing the 
(marked) root, at least one of the branches must have size one. This 
leads to the differential equation 
, 1 1 1 
H(z) = H(z)- —i = exp (In a), 


with initial condition H(0) = 0. Note that we obtained the number of 
trees that have a branch of size one by subtracting from the number of 
all trees the number of trees that do not have such a branch. 


Solving this differential equation, we get that H(z) = —1+ exp(—2) 
That shows that if n > 0, the number of such vertices is equal to the 


number of derangments of size n, which is an interesting fact. 


Let us use the exponential formula with inside function f(n) = (n— 1)! 
for n > 2 and f(2) =u. Then the inside generating function is 
2 2 n 


git z z 22(u— 1 
F(eu) = Dat =“ Fy Fa EHO (=) 


n>2 n>3 


So the combined generating function is 


exp (= ) 


H(z, u) = exp(F(z, u) = as 
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Computing dH/du(z, u), then setting u = 1, we get the generating func- 


tion h(z) = 422=. From this, it follows that hy = # Yep GP. 


77. Let i <j. Write the entries of p around a circle so that p; immediately 
follows i when moving clockwise. Let p’ be the permutation obtained 
from p by interchanging 7 and 7 around the circle. If 7 and 7 are not 
in consecutive positions around the circle, then 7 precedes 7 in exactly 
one of p and p’ when written in the one-line notation. Otherwise, if a, 
i, and j are in consecutive positions around the circle, examination of 
all cases shows that the only case when such a one-to-one match is not 
possible is when 2 < a < j. That happens for 7 —2—1 values of a, and in 
those cases, 7 will precede 7 in both p and p’. This leads to the explicit 
formula 3 ‘ 

n— 
f(n) =n), 
for n > 3. See entry A227404 and its references in [286] for more infor- 
mation about this sequence. 


( 


10.4 Solutions for Chapter 4 


1. Recall the notion of rank from Theorem 4.21. Generalize the Simion— 
Schmidt bijection of Lemma 4.4 as follows. Instead of fixing just the 
left-to-right minima, fix all entries that are of rank at most k — 2. Then 
proceed like in the Simion—Schmidt bijection. 


3. Let p be a permutation enumerated by Av,,(132,312) = a,. Let us say 
that n is in position 7. Then all entries preceding n are larger than 
all entries that n precedes. Moreover, entries that n precedes are in 
decreasing order. This proves the recursive formula ay = 5°", a;—1 for 
n > 2, with a; = 1. Solving this recursion, we get a, = 2"7!. 


5. If p avoids 132, then all entries on the left of n are larger than all entries 
on the right of n. Furthermore, the subword on the left of n has to avoid 
123 as well, while the subword on the right of n has to avoid 132 and 
1234. These conditions together are sufficient. 


The reader is invited to prove that Av,,(123,132) = 2"~1. Now let 
by = Av, (132, 1234). Let n be in position 7 of our permutation. Then 
it follows from the above that there are 2'~2b,_; ways for p to be 
(132, 1234)-avoiding if 7 > 1, and b,_1 ways if i = 1. This implies 


bp = bn-1 + 3S Oe hes 


1=2 
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Solving this recurrence relation, we get that b,, = Fo,, where F; is the ith 
Fibonacci number, that is, Fy = 0, Fy = 1, and then Fi41 = Fi + Fy-1. 
This result was proved in [327] by a different argument. We point out 
that [327] has a catalog of results for sequences Av,,(p,q), where p is of 
length three, and q is of length four, as well as a general proof technique 
to obtain those results. 


. We show that for n > 3, we have g(n) — g(n — 1) = n- 2”~%. Indeed, 


there are g(n — 1) permutations that avoid both 132 and 4231 in which 
n is in the last position. If n is not in the last position, then each entry 
of L is larger than each entry of R. Moreover, L is a permutation that 
avoids 132 and 312, and R is a permutation that avoids 132 and 231. 
These conditions are sufficient. It then follows from Exercise 3 that the 
number of such permutations for each position of n is equal to 2"~? if 
n is not in the first position, and to 2”~! if n is in the first position, 
proving our recursive formula. The statement of the exercise is then 
proved by induction. 


(a) The numbers r,_1 = Av,,(3142, 2413) are the famous large Schroder 
numbers. See [326] for a proof of the recursive formula 


Mm = 3 on ‘ Cre3: 
i=0 


It is easy to see from the above formula that the number r,, also 
counts subdiagonal lattice paths from (0,0) to (n,n) that use steps 
(0,1), (1,0), and (1,1). This proves the recurrence relation 


n 


| oe ee Hi Soa (10.4) 


i=1 


From (10.4) we see that 


5 Le 1-—6r+2 
y eo = —————; 
2x 
n>0 


(b) There are nine other pairs (p,q) that are not trivially equivalent 
to (3142, 2413), but are still enumerated by the same number of n- 
permutations. One of them is (1324, 1423) as was proved in [326]. 
Another pair was found by Sophie Gire [196]. Finally, it was Darla 
Kremer [234] who gave an exhaustive list of all ten pairs. The 
remaining eight pairs are (1234, 2134), (1342, 2341), (3124, 3214), 
$(3142, 3241), (3412, 3421), $(2134, 1324), (3124, 2314), and finally, 
(2134, 3124). We point out that the Schroder numbers occur in 
many other contexts, and the interested reader should consult 
Chapter 6 of [297] for details. 
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11. 


(a) This result was proved in [49]. Denote by h(n) = Av, (1324, 2413) 


the number of the permutations to enumerate, for brevity. It is 
obvious that if n is the leftmost entry, then the number of such 
permutations is h(n — 1). Now let p be a (1324,2413)-avoiding n- 
permutation; suppose n is not the leftmost entry of p and let a be 
the smallest entry of p that precedes n. Then n precedes the entries 
1,2,---,a@—1. Furthermore, these a — 1 entries must occupy the 
last a — 1 positions (why’). 

So the last a — 1 entries of p are the smallest ones, and so we can 
have h(a — 1) different strings on them. Let t(n — a+ 1) be the 
number of possible substrings on the first n—a+1 entries, in other 
words, t(z) is the number of (1324,2413)-avoiding n-permutations 
in which the entry 1 precedes the entry n. In what follows, we are 
going to use this second interpretation of t(n) so as to alleviate 
notation. Set t(0) = 0. Let T(z) = >0,., t(n)z”. It follows from 
the above that permutations counted by ¢(n) are precisely the inde- 
composable (1324,2413)-avoiding n-permutations. It is then clear 
that H(z) = 1/(1—T(z)), and this includes even the case when n is 
the leftmost entry. Now we analyze the structure of permutations 
enumerated by the ¢(z) in order to determine T(z). 

Call entries before the entry 1 front entries, entries after the entry 
n back entries, and entries between 1 and n middle entries. Say 
that an entry x separates two entries y and z written in increasing 
order ify <a < z. 

The front entries must form a 132-avoiding permutation, the middle 
entries must form an increasing subsequence, and the back entries 
must form a 213-avoiding permutation. Similary, no front entry 
can separate two middle entries, or two back entries in increasing 
order; no middle entry can separate two front entries in increasing 
order or two back entries in increasing order; and no back entry 
can separate two middle entries or two front entries in increasing 
order. 

Therefore, the only way for two entries of the same category to be in 
increasing order is when they relate to any entries of the other two 
categories in the same way. Such entries are said to form a strong 
block. The strong block subdivision of a permutation counted by 
T(z) is shown in Figure 10.3. 

As we said, each strong block between 1 and n consists of an in- 
creasing subsequence, while strong blocks in the front are 132- 
avoiding permutations, and strong blocks in the back are 231- 
avoiding permutations. Permutations satisfying all these condi- 
tions do avoid both 1324 and 2413. 

Now for i > 2, let vj; be the number of those permutations 
counted by t(z) containing no middle strong blocks, except for 1 
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n 


FIGURE 10.3 
The strong block subdivision of p. 


and i. So v; = 1, ve = 2, v3 =6,---. Let R(x) = >, viz" be the 
generating function for the v;. Then clearly T(z) = TRG): 

Note that v;_, is just the number of ways to partition the in- 
terval {2,3,---,i— 1} into disjoint intervals, and then taking a 
213-avoiding or a 132-avoiding permutation on each of them alter- 
natingly. It is not hard to see by a lattice path argument that this 
means that v; = (7°77), so R(z) = so. Therefore, 


i-l 
2 zV1—4z 


Cel ey ae 


which implies 


~ 1-T) (1 —42)(1— 22-2 
1—524+3224+ 22/1 — 4z 


1— 6z + 8z2 — 423 


He 1 (/T— az — z)(./T— 4a(1 — 2) +2) 


There are five nontrivially equivalent pairs (p, g) so that Av, (p,q) = 
h(n) for all n. Of the remaining four, the pair (1324, 2143) was 
found in [209], though a more accessible reference is [89]. (We 
note that permutations avoiding this pair are called smooth per- 
mutations. The pair (3214, 4123) was found in [288], and the pairs 
(1342, 2314) and (1342, 3241) were found in [49]. 


— 
a 


13. This problem has been solved in [207], where the authors showed that 


1—z—-V1—2z2-32 
LCs a ae a ec (10.5) 
n>0 
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FIGURE 10.4 
A 1-2 tree. 


15. 


The numbers [,,(2143) are called the Motzkin numbers, and occur in 
numerous combinatorial problems. See [297] for an extensive list of 
these problems. 


The authors of [207] showed the above formula by finding a bijection 
between these involutions, and 1-2 trees. A 1-2 tree is a plane tree 
in which each vertex has 0, 1, or 2 children. However, in contrast to 
decreasing binary trees, if a vertex has a single child, that child is directly 
below its parent, not on its left or right. See Figure 10.4 for an example. 


It is not difficult to see that the numbers of these trees satisfy the re- 
currence relation My = Mn—1 + 222) MypMn—2—r, from which (10.5) 
follows for I,(2143) = My. 


As a 132-avoiding permutation is skew indecomposable if and only if 
it ends with its largest entry, the number of such k-permutations is 
Cy-1. Let our three blocks end in positions 7, 7, and n. With these 
restrictions, we clearly have C;_1Cj—;-1Cn_j-1 permutations with the 
desired property. Now we have to sum this expression over all possible 
t and j to get the total number of 132-avoiding n-permutations having 
exactly three skew blocks. We do this in two steps. First, fix 7, and 
compute the sum 


n-1 
Ci-1 ) Cy—i1-1Cn—j—1 = Ci-1Cn_-i-1.- 


jit 


Second, we sum over all possible 7 to get that there exist 


n—2 
S Ci 1Cn a 1=C, 1-Ch 2 
i=1 


permutations with the desired property, as long as n > 3. 
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Let p be a 231-avoiding n-permutation. Let us call p decomposable if it 
can be cut into two parts so that each entry before the cut is smaller than 
each entry after the cut. For example, 21543 is decomposable. If p is 
decomposable, then say that its first cut is after 7 entries (1 <i<n-—-1), 
and that p = LR, where L is the substring of the first ¢ entries. Then 
we define the northeastern lattice path f(p) recursively, by taking the 
path f(L) from (0,0) to (,7), and continuing it with a translated copy 
of the path f(R) from (i,7) to (n,n). 

We still have to define f for 231-avoiding permutations that are not du- 
ally decomposable. These permutations start in their entry n (otherwise 
they would have a cut immediately before the entry n). Let p be such 
a permutation, and let p’ be the (n — 1)-permutation obtained from p 
by omitting the entry n. Then define f(p) as the concatenation of the 
step (0,0) to (1,0), a translated copy of f(p’) from (1,0) to (n,n — 1), 
and the step (n,n — 1) from (n,n). 


It is straightforward to prove again by induction that this recursively 
defined map f is indeed a bijection. By induction again, the number of 
ascents of p is equal to the number of north-to-east turns of f(p). 


Yes. If p is increasing and q is decreasing, then all permutations of 
length at least |p| - |q| + 1 must contain at least one of p and q. This is 
a famous result of Erdés and Szekeres, and we proved it in Proposition 
6.52. 


As 231 and 312 are inverses of each other, and the inverse of an even 
permutation is even, the first equality is straightforward. 


A permutation is 231-avoiding if and only if its reverse is 132-avoiding. 
On the other hand, reversing a permutation is the same as multiplying 
it with the transpositions (1), (2n—1),---. The number of these trans- 
positions is |n/2], and our proof follows from the result of the previous 
exercise. This result first appeared in [284], in a slightly different form. 


We claim that for n > 1, Av,(123, 132,213) = F,41, where the F,, are 
the well-known Fibonacci numbers, starting with Fo = 0, and F, = 1, 
and then given by F,41 = F, + Fn_-1. This result was first mentioned 
in [284]. Indeed, in a permutation p enumerated by Av,,(123, 132, 213), 
the entry n must be in either the first or second position; otherwise, p 
could not avoid both of 123 and 2138. If n is in the first place, then, by 
induction, we have F,, possibilities for the rest of p. If n is in the second 
place, then n — 1 must be in the first place; otherwise, a 231-pattern 
is formed. Then, by induction again, we have F;,_1 possibilities for the 
rest of p. This shows that Av,,(123, 132, 213) = F,41. It is well-known, 
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25. 


27. 


29. 


and can be proved by routine generating function techniques, that 


Hoel iw a, eae 
Cae 2 5 D 


As (54) < 1, the first term is dominant, showing that we have 


°/Kv,, (123, 213, 132) = (+54). 


The pattern 132456---k can play the role of g. The proof is similar to 
that of Theorem 4.29. However, instead of simply defining a strong class 
by its left-to-right minima and right-to-left maxima, we also have to fix 
the value of the position of entries that become right-left maxima if the 
original right-to-left maxima are removed. We then have to iterate this 
procedure n — 4 more times. 


(a) Let p be a 132-avoiding n-permutation in which the entry n is in 

position 7. Then the binary plane tree T(p) will have a left subtree 
of « — 1 nodes and a right subtree of n — 7 nodes. The rest of the 
tree is constructed recursively by the same rule. The map T is a 
bijection as the position of n in p can be read off T(p) as the size of 
the left subtree of T(p) plus one. The position of the other vertices 
can then be found recursively, noting that if n is in position 7, then 
the set of entries of the left of nm must be {i+ 1,i+2,---,n}, and 
the set of entries on the right of n must be [i — 1]. It is here that 
we use the fact that p is 132-avoiding. 
Note that while T(p) is an unlabeled tree, each node of T'(p) is nat- 
urally associated to an entry of p. Nevertheless, T'(p) is unlabeled 
as writing these entries to the nodes would not carry any additional 
information. 


(b) It is straightforward to prove by induction that d(p) is the number 
of right edges (that is, edges that go down and right, or, if you like, 
from northwest to southeast). 


(c) We claim that p; > pj+1 if and only if the vertex corresponding to 
pi appears on a higher level in T(p) than the vertex corresponding 
to pj41. This is obvious if n = 2. Now assume our claim is true 
for all integers less than n. Then if p; and p;;; are on the same 
side of the entry n, then the corresponding vertices are in the same 
(left or right) subtree of T(p), and our claim follows by induction. 
Consecutive entries cannot be on two different sides of n, so the 
only remaining case is when one of p; and p;+1 is equal to n. That 
case is trivial, however, as that vertex will correspond to the root 
itself. 


The number of 1234-avoiding n-permutations is equal to the number 
of strong classes of n-permutations. On the other hand, the number of 


452 


3l. 


33. 


35. 


Combinatorics of Permutations, Third Edition 


1324-avoiding n-permutations is asymptotically more than that. Indeed, 
for n > 7, it is very easy to construct strong classes that end in the class 
3 * 1 * 7 * 5 by concatenation. These classes contain at least two 1324- 
avoiding permutations. On the other hand, they constitute at least a 
constant factor of all strong classes, implying our claim. 


Let p be a pattern of length k& starting with 1 and ending with k so that 
Avn(p) < C” holds for all n, for some constant C. Then we claim that 
Avn(q’) < (4cC)", thus we can set K = 4cC’. Take an n-permutation 
ma which avoids gq’. Suppose it contains q (this will only exclude c” 
permutations). Then consider all copies of g in our permutation and 
consider their entries x. Color these entries red. Clearly, the red entries 
must form a permutation which does not contain p. Suppose they do, 
and denote x; and xx the first and last elements of that purported copy 
of p. Then the initial segment of the copy of q which contains x; and 
the ending segment of the copy of gq which contains x, would form a 
copy of q’. 

Now remove all the red entries from 7, to get 7’. Then x’ must be 
q-avoiding as all copies of q in z lost their entries playing the role of zx. 
Therefore, there are at most c” possibilities for the permutation of the 
non-red entries. There are at most 2”~! choices for the positions of the 
red entries, at most 2”~! choices for the values of the red entries, and at 
most C"~! choices for the permutation of the red entries. This shows 
that less than (4C)"~!-c" +c” < (4Cc)" permutations of length n can 
avoid q/. 


It suffices to prove that every q-avoiding n-permutation p can be ex- 
tended into a g-avoiding (n + 1)-permutation by prepending it with a 
new first entry, in & — 1 different ways. Let 7 be the first entry of q. 
Then we can prepend p with any one of the entries 1,2,---,i— 1, as 
well as any one of the entries n,n—1,--- ,n—k+i+1 without creating 
a copy of q. Indeed, the first entry of the obtained (n + 1)-permutation 
would be either too small or too large to play the role of 7 in any copy 
of g, and, being the leftmost entry, it certainly cannot play the role of 
any other entry of q. 


These will be the trees in which going from the leaves up, each label 
is as large as it can possibly be, that is, the sum of its children plus 1. 
Indeed, in 123-avoiding permutations, the entries that are not left-to- 
right minima form a decreasing sequence, meaning that each vertex will 
contribute to its own label. 


Note that this observation provides a bijection from the set of indecom- 
posable 123-avoiding n-permutations to that of unlabeled plane trees on 
n vertices. 
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FIGURE 10.5 
The area between L and L' = f (53124) is equal to (53124) = 6. 


37. 


39. 


Al. 


The number of inversions is translated to an area. More precisely, let L 
be f(123---n), the staircase lattice path. Let f(p) = L’ for some 231- 
avoiding n-permutation p. Then i(p) is equal to the area between L and 
L’. See Figure 10.5 for an example. This can be proved by induction, 
using the block decomposition of p explained in the solution of Exercise 
17. 


We mention that the area statistic of these permutations leads to an 
interesting open problem. Let n be fixed, and let az be the number of 
231-avoiding n-permutations p so that the area below f(p) is equal to k. 
Then it is conjectured in [304] that the sequence {a,} is unimodal. It 
is also conjecture in [304] that unimodality remains true if the staircase 
Ferrers shape that f(p) is not allowed to enter is replaced by another 
self-conjugate Ferrers shape. 


Yes. We claim that for k > 2, we have Av,,(123,(k — 1)k---21) < 
n2(k—-2) We prove this claim by induction on k. For k = 2, the statement 
is trivial, and for k = 3, the statement is true as Av,,(123,231) = 
(BS) +1<n?. 

Now let us assume that the statement is true for k, and prove it for k+1. 
It is clear that if a permutation p avoids both 123 and (k—1)k--- 21, then 
the substring p’ obtained from p by omitting its right-to-left minima 
must avoid both 123 and (k — 2)(k — 1)---21. Note that p can have 
at most two right-to-left minima as p is the union of two decreasing 
sequences. One of these two right-to-left minima must be the entry 1, 
and the other is the rightmost entry. We have at most n choices for 
the position of n, at most n choices for the rightmost entry, and, by 
induction, at most n2(*-8) choices for p’. Therefore, we have at most 


n-n- n2k-3) = 72(k-2) Choices for p as claimed. 


It is proved in [3], along with many similar results on multiset permu- 
tations avoiding patterns of length three, that 


By (a1, a2, a3) = (" ie a = é -) + « eats ) 


a3 a3 a2 
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Let a, be the number of 1s contained in A,. Then ap = 1, and an41 = 
2an + 2” for n > 0. So, if A(z) = >7,,59 anz”, then we have 


1 z 
—— ————— =] 9)9n-1 nr 
[22  (—oe = Nes oe 


A(z) 
So dn = (n+ 2)2"~1 if n > 1 (and the rule works even for n = 0). Note 
that A, does not contain B. If m = 2", then this means that there 
exists a matrix of size m x m that contains more than (m/2) - log, m 
entries equal to 1 that avoids B. 


This result is due to Sergi Elizalde and Marc Noy [158]. 


If n = 2m +1, then this number is the Catalan number C,, and if 
n = 2k, then this number is the Catalan number C;,. These facts can be 
proved by considering the position of the entry 1 of our permutations, 
and then recognizing that the numbers of our permutations satisfy the 
same recurrence as the Catalan numbers. 


If p avoids q, then p~! © p is an involution that avoids q. 


Let p avoid Hy, and let p; be the leftmost entry of p that is of rank k—1. 
Then the string pipo---pi-1 is a 12---(k — 1)-avoiding permutation, 
while the string p;--- pp is an increasing sequence. This yields the chain 
of inequalities 


n 


Ava(H) < )( . Jaa 


i=1 


< ((k-1)? +1)” 
We used the binomial theorem in the last step. 


(a) Let skew indecomposable 132-avoiding permutations correspond to 
Dyck paths that never touch the horizontal axis, except in their 
starting and ending point. Then define f recursively, noting that 
skew indecomposable 132-avoiders must end in their largest en- 
try, while their images have the property that removing the origi- 
nal (1, 1)-step and the ending (1,—1)-step, we get a Dyck path of 
semilength n — 1. 

(b) It is easy to prove that p has a ULIS if and only if f(p) has a 
unique peak of maximal length. Furthermore, p is an involution 
if and only if f(p) is symmetric to the vertical line x = n. So if 
p € UTI,,(132), then f(p) is symmetric to the line z = n and has a 
unique peak of maximal length (which is necessarily on that line). 
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Let p be a permutation of length n > (k — 1)? +1 that avoids the 
pattern 12---k. Then p is the union of & — 1 decreasing subsequences. 


Color these subsequences with colors 1,2,...,&—1 so that a maximum- 
length subsequence gets the color 1. Then there are at least (k —1)?+1 
entries of color 1, say in positions 4; < ig < +--+ < im, and they are 


p(i1) > pli2) > +++ > plim). Let Pj = p(i;). 

Consider the m > (k — 1)? +1 positions Py, < Pm—1 < ++: < Py in p. 
There will be a set S of at least k positions among them so that entries 
in these positions are of the same color, that is, that form a decreasing 
subsequence . Those same k entries form an increasing sequence in p?. 
Indeed, let positions P, and P, contain two entries of S', with a < b, so 
P, > P,. That means that position p(i,) contains a smaller entry than 
position p(i,), that is, p(p(ta)) < p(p(i»)). Repeating this argument for 
every pair of entries in positions that belong to S proves that in p*, the 
k entries that are in positions in S form an increasing subsequence. See 
[79] for more on this problem. 


It is proved in [80] that Cyc,,(123, 132) = 2\("—)/?J, 


Let p = pip2--- Pn be any cyclic permutation of length n that avoids q. 
Insert the entry n+ 1 to the next-to-last position of p. Then p is still 
q-avoiding, since n + 1 is too far back in p to be part of any copies of 
q. Furthermore, the obtained permutation p’ is still cyclic, since p; = p’, 
for alli <n—1, and p maps n to x, while p’ maps n to n+ 1, and then 
n+1to x. So, we get the cyclic diagram of p’ by simply inserting the 
entry n +1 between n and z in the cyclic diagram of p. 


Doing this for all Cyc,,(q) cyclic, g-avoiding permutations of length n 
yields a set S of Cyc,,(q) cyclic q-avoiding permutations of length n+ 1, 
each of which contains the entry n+ 1 in the nth position. As q is 
an involution, the inverse r~! of any g-avoiding permutation r is also 
q-avoiding. So taking the inverse of each permutation in S yields a set 
T of Cyc,,(q) cyclic q-avoiding permutations of length n+ 1, each of 
which contains the entry n in the n+ 1st position. Finally, S and T 
are disjoint sets, since a cyclic permutation that is longer than 2 cannot 
contain the 2-cycle (n n+ 1). This result was first proved in [80]. 


Let C = Av(12---(k + 1), 132, 213). If p € C, Then the entry 1 can be 
only in one of the last k positions. Furthermore, all entries on the right 
of 1 must be smaller than all entries on the left of 1, and the entries on 
the right of 1 must form an increasing sequence. It is now easy to set 
up a bijection from C, to the set of all compositions of n into parts at 
most k. The length of that last increasing subsequence starting at 1 will 
correspond to the last part of a composition, then the other parts are 
recursively defined by this same process. (Alternatively, the numbers of 
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both kinds of objects satisfy the recurrence relation f(0) = 0, f (i) = 2* 
for i € [k], and f(n) = 7*_, f(n—1) ifn >k. ) 


10.5 Solutions for Chapter 5 
1. We claim that Sj32,1(n) = Co). See [47] for a proof. The main idea is 


the following. Ina penmutadon enumerated by 5132,1(7), there is either 
one or no front entry that is smaller than a back entry. If there is one 
such front entry, then its position and size is very restricted. If there 
is no such front entry, then the single 132-pattern of the permutation 
is formed either by front entries only, or by back entries only. This 
leads to a recurrence relation involving the numbers $j32,1(n) and the 
Catalan numbers, and solving that recurrence relation, we obtain the 
above explicit formula. 


3. This classic problem was first solved by Cayley [111], who proved that 


1 n+d4+2\/f/n-1 
Flo) = ——( d+1 )( d ), 


Recently, Richard Stanley [296] gave a proof based on a bijection be- 
tween these polygon dissections and Standard Young Tableaux. We will 
revisit that proof in Exercise 22 of Chapter 7. 


5. We prove the statement by induction on k. For k = 1, we have S(n, 1) = 
1, which is obviously P-recursive. Now let us assume the statement is 
true for k —1, and prove it for k. We have seen in Exercise 8 of Chapter 
1 that 

S(n,k) —kS(n—1,k) = S(n—1,k-1). 
Let Sx (z) = 30,5, 5(n, k)z”. Multiplying both sides of the above equa- 
tion by 2”, and summing over all n > k, we get 


Sp(z)(1 — kz) = zSp_1)(z) 
2Sp-1(2) 
1—kz ’ 


so S;,(z) is the product of the d-finite generating functions zS,_1(z) and 
and is therefore d-finite. 


Si (z) = 


1 
1—kz? 
Note that it follows from the above that 


gk 


Sel) = Ga aay a he) 
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Let A(z) be algebraic of degree a, and let B(z) be algebraic of degree 
b. Then A(z)* can be obtained as a linear combination of the power 
series 1, A(z), A(z)?,--- , A(z)*~' in which the coefficients are polyno- 
mials. The analogous statement holds for B(z)’. That means that any 
expression involving A(z) and B(z) and algebraic operations on them 
can be obtained using linear combinations of the A(z)’ and B(z)), with 
it <a—land J < b—1, and the same algebraic operations that are used 
in the desired expression. 


. On one hand, f”(z) = —f(z), so the dimension of the vector space 


spanned by the derivatives of f is at most of dimension two; therefore, 
f is d-finite. On the other hand, assume that f is algebraic of degree d, 
that is, 

Po(z) + Py(z)sin(z) +--+ + Pa(z) sin?(z) = 0, 


with d being minimal. As P,(z)sin(z) + --- + Pi(z)sin’(z) = 0 for 
infinitely many values of z, it follows that P5(z) must be the zero poly- 
nomial. So the above equation reduces to 


sin(z)(P,(z) +---+ Pysin¢—1(z)) =0. 


As sin(z) is not identically zero, it follows that P;(z)+---+Pisin? !(z) = 
0 as a function, contradicting the assumption that the degree of sin(z) 
was d. 


Note that this proof depended on the fact that sin(z) has infinitely 
many roots. See [297], Chapter 6, for two proofs of the fact that e* is 
not algebraic. On the whole, however, it is often difficult to prove that 
a series is not algebraic. 


It is proved in Exercise 11 of Chapter 4 that 


1—524+ 322+ 22/1 —4z 


H(z) = 5> Avn(1342, 2431)2” = 1 — 6z + 822 — 423 


n>0 


In other words, the generating function H(z) of our sequence is alge- 
braic. Indeed, if a(z) = 1 — 5z + 32? and b(z) = 1 — 6z + 82? — 423, 
then (b(z)H(z) — a(z))? = 24(1 — 4z), and the algebraic property of 
H(z) follows. Therefore, H(z) is d-finite, proving that our sequence is 
P-recursive. 


A permutation is layered if and only if it avoids both 312 and 231. On 
one hand, the condition is necessary, since in a layered permutation, any 
purported copy of any of those two patterns would have to contain the 
entries playing the roles of 3 and 1 within the same descending run, and 
that would make it impossible to find an entry to play the role of 2. 


On the other hand, in permutations avoiding 312, the entries after the 
maximal entry have to be in decreasing order, and in permutations 
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A grid for a k-superpattern. 


15. 


17. 


19. 


avoiding 231, the entries after the maximal entry have to be larger than 
entries before the maximal entry. So, in permutations avoiding both 
312 and 231, entries weakly after the maximal entry must form the 
subsequence n(n — 1)---(n — k) for some k. Iterating this argument 
completes the proof. 


Induction on n. The claim is clearly true if n < 2. Assuming it is true 
for n — 1, first consider c < Co) and let p € S,_1 satisfy the lemma. 
Then pn € S,, works for such c. On the other hand, if ("5") < ¢ < (3) 
then consider c! = c— (n—1)< eur Pick p € S,_1 with c’ copies of 
21 and none of 132. Then np € S,, is the desired permutation. 


A 321-avoiding n-permutation consists of two increasing subsequences, 
of sizes a and n—a. Therefore, the number of inversions in p is at most 


a(n—a) < oe This is indeed attainable, for permutations like 456123. 


(a) We claim that sp(3) = 5. On one hand, 41352 is a 3-superpattern. 
On the other hand, there is no 3-superpattern of length four as a 
pattern of length four can contain at most four different patterns 
of length three. 


(b) The permutation 1 3610259487 is a 4-superpattern. 

(c) Consider the grid shown in Figure 10.6. 
Then read the columns one after another starting with the leftmost 
one, going from the bottom up in each column. The obtained 
permutation is clearly a k-superpattern. Indeed, if we want to find 


a copy of gq = qi--- qx, all we need to do is to choose the q;th entry 
of the ith column for each 7. 


(a) A permutation of length six has ee = 15 subwords of length four, 
so it cannot contain all 24 different patterns of length four. 


b) A 4-superpattern has to contain both the increasing and the de- 
g 
creasing pattern of length four. A permutation p of length seven 
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29. 
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33. 


can only do that if it is the union of a 1234-pattern and a 4321- 
pattern, which intersect in exactly one entry. If p has this decom- 
position property, then all of its subwords can be decomposed into 
an increasing and a decreasing sequence. Therefore, 3412 cannot 
be a subword of p. 


For any k, the number of different k-element patterns contained in p is 
at most min((?) ,k!), as there are G) subwords of p that have k elements, 
and there are k! different possibilities for the pattern of these subwords. 
One checks that (?) < k! if and only if k < 5, so we get that the number 
of different patterns in p is at most 1+1+2+6+24+56+28+8+1 = 127. 


No, this is not always true. As is shown in [268], the case of k = 6 
provides the smallest counterexample. In that case, the layered permu- 
tation having seven layers of length n/7 each will have more copies of q¢ 
than the permutation having six layers of length n/6 each. 


This result and its proof can be found in [4]. 


Set Mo = 1, and M(z) = 0,39 Mnz”. The number of such paths that 
first return to the line y = 0 at the point (k,0) is clearly My_2Mn_p if 
2<k<n, and M,_; if k = 1. This leads to the functional equation 


M(z)=14+ M(z)z+ M?(z)z?, 


hence 


1—z—vV1—2z-— 32? 


In other words, M(z) is algebraic, and therefore, d-finite. 


On one hand $ = 1232123 contains all six permutations of length three, 
hence, allperm(3) < 7. On the other hand, let us assume that there 
exists a sequence Z of length six of that property. If there is an entry, 
say 1, that occurs in Z only once, then there would have to be at least 
three entries before that entry 1, and at least three entries after that 
entry 1 in order for Z to contain all four permutations starting or ending 
in 1. That is impossible, since Z has only six entries. So Z must contain 
two copies of each of 1, 2, and 3. Let us assume without loss of generality 
that Z starts with 1. Then the other occurrence of Z cannot be in the 
second, third, fourth, or sixth positions, for obvious reasons. So the 
second occurence of 1 is in the fifth position. Then Z cannot contain 
both 312 and 213, since the sixth entry of Z cannot be both 2 and 3. 
This proves that allperm(3) = 7. 


Let Z be a sequence of length allperm(n) that contains all n! permuta- 
tions of length n as a subword. Let i be the element of [n] that shows 
up lastin Z. That means that the leftmost copy of 7 in Z is in position 
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n or later. Note that Z contains all n-permutations, so in particular, 
Z contains all n-permutations starting with i. So if we remove all el- 
ements of Z that either precede the first occurrence of 7 in Z, or are 
equal to 7, we get a shorter sequence Z’ whose elements come from an 
(n — 1)-element set and that contains all (n — 1)! permutations of that 
set as a subword. As Z’ is at least n letters shorter than Z, this implies 
that allperm(n) > allperm(n — 1) +n, after which our claim is routine 
to prove by induction. 


In fact, the generating function of the sequence is not only d-finite, but 
also rational. This is easy to prove by induction on k. 


This result has been first proved by Daniel Ashlock and Jenett Tillotson 
[18]. The proof is by induction. 


Take an n-superpermutation p. Start moving left to right. When you 
find the first factor q that is equal to a permutation of length n, replace 
that copy of q by the string q(n + 1)g. This adds n + 1 new letters to 
p. Continue reading, from the first digit of the second copy of qg in that 
newly added string g(n + 1)g. When you find an n-permutation that 
you did not encounter before, stop, and repeat the above procedure. 


As p contained all n! permutations of length n, we will carry out the 
above procedure n! times, adding a total of (n+ 1)n! = (n+ 1)! new 
letters. 


The obtained long string is an (n + 1)-superpermutation because it con- 
tains the permutation L(n + 1)R as a factor. Indeed, at some point 
during the above procedure, the permutation RL will be replaced by 
the string RL(n+ 1)LR. 


For instance, for n = 2, we can start with the 2-superpermutation 121, 
and get the 3-superpermutation 123121321. 

Note that this argument proves that if spp(n) < m, then spp(n + 1) < 
m-+(n+1)! 


(I 


10.6 Solutions for Chapter 6 


1. 


(a) The choice of i, 7, k, and @ is clearly insignificant. Applying the 
Transition Lemma, we see that the four entries n — 3, n—2,n—1, 
and n are in the same cycle of p if and only if n is the leftmost 
of the four of them in f(p). This obviously happens in 1/4 of all 
n-permutations, so the probability in question is 1/4. 


(b) The entries n — 3, n — 2, n—1, and n belong to different cycles 
of p if and only if they are in increasing order in f(p). The latter 
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happens in 1/24 of all n-permutations, so the probability we are 
looking for is 1/24. 


3. We can again assume that the four entries in question are n — 3, n — 2, 
n—1, and n. 


(a) This will happen if and only if the pattern formed by our four 
entries has two left-to-right maxima. There are c(4,2) = 11 such 
permutations, so the probability we are looking for is 11/24. 


(b) Similarly, this will happen if the pattern of our four entries has 
three left-to-right maxima. The number of such permutations is 
c(4, 3) = 6, so the probability in question is & = 4. 

5. This exercise is very similar to Example 6.27. Let S be a k-element 
subset of [n], and let X(p) be the number of k-cycles of p. Then the 
probability that the entries belonging to S form a k-cycle is (k — 1)!- 
ee therefore, this is the expectation of the corresponding indicator 


Wile 
n 


variable Xg. As there are ( ie) such indicator variables, we get that 


-_ _ (n (n—k)l 1 
BE) = Dee) (;) 1 Sod 


7. For obvious symmetry reasons, we have E(Z) = (3)/2. To compute 
E(Z?), introduce the indicator variables Z;,; defined for p = pip2--- Pn 
and for all pairs of elements 7 < j by 


1 if pj > p,; , 
Zi,j(P) = e if not. 

It is then clear that Z = )),_, Zi,;. Furthermore, E(Z;,;) = E(Z?;) = 
1/2. There are several cases to consider when computing the expecta- 
tions E(Z;,; Zx,c). The simplest is the case of the (3) ("s3) /2 pairs when 
{i,j} and {k, 2} are disjoint. In that case, clearly E(Z;,;Z%,¢) = 1/4. 
Ifi =k, but 7 # 4, then E(Z;,;Z4,¢) = 1/3 as two of the six possible 
patterns for the triple p;p;pe are favorable (the ones in which p; is the 
largest of the three entries). Similarly, we have E(Z;,;Zp,¢) = 1/3 if 
j = but i#k. Each of these possibilities occurs in (2) cases. Finally, 
it can also happen that j = k, in which case E(Z;,;Zx,¢) = 1/6 (the pat- 
tern p,p;p-ell has to be decreasing) or that 7 = @, in which case again, 
E(Z,,;Zk,2) = 1/6 (the pattern pypip; has to be decreasing). Each of 
these scenarios occurs (3) times. Therefore, we have 


E(2)=S°E(Z2,)+ So E(Zi Zp) 


tJ GI)A(KE) 


nau) a) ta) 


I 
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and consequently, 


Var(Z) = 


| 
& 
oe 
& 
S 


| 
Dole 
a 
Nw 3 
Sey 
+ 
ALR 
i 
N38 
Sy 
os 
3 
Nn | 
bo 
eed 
wl] or 
oN 
ws 
NS 

| 
3 
bo 
aT 
|S 
my | 
— 
mre 


. No, there is not. If there were, applying the hooklength formula for such 


a Ferrers shape, the numerator, being 20!, would be divisible only by 
77, while the denominator would contain three factors equal to 7. 


We prove that we can even create a tree that consists of a single path 
that satisfies at least one-third of the constraints. 


These trees correspond to permutations of [n]. The generic constraint 
{(a, b), (c, d)} will be satisfied by eight of the possible 24 relative orders 
of these four elements as we can swap entries within the pairs, or we can 
swap the pairs. Let p be a random n-permutation, and let X(p) be the 
indicator variable of the event that the tree defined by p satisfies the 
constraint X. Then E(X) = 1/3, where the expectation is taken over all 
n-permutations p. Using the linearity of expectation for the indicator 
variables of all constraints, we obtain our claim. 


Let R be the range of X. By the definition of expectation, we have 


w= E(X) =) iP[X =) = So iP[X =4+ DO iP[X =i] 


iER i>ap i<ap 


Dividing by ay, we get Markov’s inequality. 


As p is an involution, its cycles are all of length one or length two. Let 
I, denote the number of involutions of length n. Then the number of 
involutions of length n in which the entry 1 is fixed is [,_1, whereas 
the number of involutions of length n in which the entry 1 is part of a 
2-cycle is (n — 1)In—2. This shows that 


In-1 + 2(n = 1)In-2 


E(Y) = L 


Finally, as it is easy to compute by the Exponential formula, or directly, 
Tn = dott?! (2) (24 — 1)!!, where we set (—1)!! =1. 


By symmetry, we have 


Pio -5(5) >a(3)| 3 xP| , 
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We have seen in the solution of Exercise 7 that if Z(p) = i(p), then 
Var(Z) = (3) 2242. Therefore, o = \/Var(Z) = O(n?/?), and our state- 
ment follows from Chebyshev’s inequality by setting \ = ca,/n for an 
appropriate constant c. 


If n is in a cycle of length more than 2, then deleting n, we get a 
derangement of length n — 1. If n is in a 2-cycle, then deleting its cycle, 
we get a derangement of length n — 2. This leads to the formula 


(n — 1)D(n — 1) 
D(n) 


(n — 1)D(n — 2) 


EXY,) = D(n) 


-EB(Yn-1 +1) 4+ E(Yn—2). 


This fact was published in [125] without proof. Let r be an n-permutation 
that consists of one cycle, and let Z, be the indicator variable of the 
event that the vertices of Gp,, form a Hamiltonian cycle in the order 
given by r. Then 


n 


“~ (n\ (n—k)! F! 1 
Bar) =) (7) POE = 
k=0 k=0 
nti 
~~ nl 


Indeed, first choose the k edges that will come from the permutation p, 
which specifies the values of p(i) for & distinct values of i. Then choose 
the remaining n — k values of p in (n — k)! ways. Similarly, the n — k 
edges of the Hamiltonian cycle r that come from q specify the values 
of q(t) for k distinct values of i, then choose the remaining values in k! 
distinct ways. Using linearity of expectation, we get that 

HA =the", 


nr 


The number of total cycles of p is equal to the number of its 1-cycles, 
plus the number of its 2-cycles, and so on. If X;,(p) denotes the number 
of k-cycles of p € S;,, and X(p) denotes the numbers of all cycles of p, 
then it follows from the result of Example 6.3 that 


E(X)=S0 F(X) = >> -. 
k=1 k=1 


Our proof will be by induction on n. For n = 1 and n = 2, the statement 
is vacuous, and it is straightforward to check that the statement is true 
for n = 3. Indeed, in the only nontrivial case, we get Aj,, = Ao,, = 1/2. 
Now assume that the statement is true for n — 1, and prove it for n. 
Then in any SYT on A, the entry n is in one of the inner corners of 
X. The presence of does not have any influence on the occurrence of 
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Aj, if i < n—1, so by the induction hypothesis (after removing the box 
containing n), we get that Ay, = Ao, =--: = An—2,,. So our claim 
will be proved if we show that An_2, = An—1,,. 

Let X be the set of SYT on A in which n — 2 is a descent but n— 1 is 
not, and let Y be the set of SYT on A in which n — 1 is a descent but 
n—2is not. It clearly suffices to show that X and Y are equinumerous; 
we do so by presenting a bijection g: X — Y. 


Let T © X. We consider three separate cases. 


(a) When n — 1 and n are in the same row, then we define g(T) by 
swapping n — 1 and n—2 in T. Note that n — 2 and n will be in 
the same row of g(T). 

(b) When n-— 1 is in a row that is strictly below the row containing n, 
and n — 2 is in the same row as n, then define g(T') by swapping 
n—1andninT. Note that n — 2 and n— 1 will be in the same 
row of g(T). 


(c) In all other cases, n — 2, n — 1, and n are in all different rows. 


(cl) First, consider the case when n — 2 is the north neighbor of 
n—1. In this case, swap n— 1 and n to get g(T). So in g(T), 
the entry n — 2 will be the north neighbor of n. 

(c2) If n—2 is the north neighbor of n, then swap n—1 and n again 
to get g(T). So n — 2 is the north neighbor of n in g(T). 

(c3) If m—2 is an inner corner, then from top to bottom, the order 
of our three maximal entries is either n, n — 2, n—1, or n—2, 
n,n—1. We then obtain g(T’) by ordering them, respectively, 
n—-1,n-—2,n, andn—1,n, n—2. Note that all three entries 
are in inner corners in g(T). 


Note that in all three subcases, the three maximal entries are in 
different rows of g(T). 


This completely defines the map g: X > Y. To see that g is a bijection, 
we show that it has an inverse. Let U € Y. By our remarks at the end 
of each case, we can establish (from the positions of the three maximal 
entries in U) which rule was used to create U, and our statement is 
proved. 


We mention that [297] contains a non-induction proof of the result we 
have just proved. 


Note that p has ri rising sequences if and only if p~! has ri—1 descents, 
or ri ascending runs. So replacing ri(p) by d(p) + 1 in the result of 
the previous exercise, we get the probability that p~! is obtained by 
our shuffle. If we sum that equation over all p € S,,, then we get 1 on 
the left-hand side. Multiplying both sides by a”, we get the identity of 
Theorem 1.8. 
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As for any X C [n], all entries of X must have the same chance to be the 
index of the minimum element of f(X), the size of F’' must be divisible 
by the size of X, and this has to hold for all possible X. Therefore, 
|F'| has to be divisible by the least common multiple of the numbers 
1,2,3,:--,n. This argument is from [100]. The authors then mention 
the well-known number theoretical fact [14] that this least common di- 
visor is of size e"—°™), 


Let X; be the indicator random variable of the event that p; +1 = pj+1. 
Then E(X;) = qt=4y = 4. Therefore, E(X) = Diy F(X) = (n- 
1)/n. 


This follows from the fact, proved in Exercise 43 of Chapter 1, that 
ascents, descents, and plateaux satisfy the same recurrence relation, 
and therefore are equidistributed. 


Let f(z) = p_, e(n, k)z*, and compute f’”(1) = 77_, k(k—1)e(n, k). 
(Note that f’(1) was computed in Exercise 65, when we computed 
E(X) = op_, ¢-) Using the product rule of derivatives, we get that 


tA 


where 1 < i,j <n. This implies that 


E(X(X-1))= ry 


=1 k=1 
So 
Var(X) = E(X?) — E(x)? 
= E(X(X -—1)) + E(X) — E(X)? 
Sab gee 1D 
a ea 


(a) Let T(z) denote the ordinary generating function for the total num- 
ber of leaves of all such trees, so T(z) = 2+ 27+32z3+---. In other 
words, T(z) counts all such trees with a leaf marked. Removing the 
root of such a structure, we get a sequence of rooted plane trees, 
one of which has a marked leaf. (With the exception of the case 
when the original tree had just one vertex, that vertex was both the 
root and the marked leaf.) The obtained trees without a marked 
vertex are rooted plane unlabeled trees, and such trees are counted 
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by the generating function C)(z) = zC(z) = (1— V1 — 4z)/2. This 
leads to the functional equation 


= % z 2) =2 ae 
T(z) =z+T( dE PEM eS Oey 
42T(z) 


—— oo a 


(1+ JI — 42)? 
Solving this equation for T(z), we get 


z 


J1 — 4z 


This shows that for n > 2, the total number of leaves is eure? 


z 1 
Ta@=5+5 


The total number of vertices in all such trees of size n is of course 
NCp—1 = (a. So ap = 1/2, meaning that half of all vertices are 
leaves. 


(b) Such trees on n vertices are in natural bijection with 132-avoiding 
permutations of length n — 1. Leaves correspond to left-to-right 
minima. So for n > 1, half of all entries are left-to-right minima in 
such permutations. 


For more information about this topic, the reader may consult the paper 
[126] of Keith Copenhaver. 


Let T(z) be the exponential generating function of the numbers T,,, with 
T(0) = 0. Removing the root of such a tree, we get a sequence of such 
trees. This leads to the differential equation T’(z) = 1/(1—T(z)). This 
yields T(z) = 1—/1 — 2z, and that leads to the formula T,, = (2n —3)!! 
forn > 1, and T, = 1. 


Let A, denote the total number of leaves in all non-plane 1-2 trees on 
vertex set [n]. Let A(z) = 735 An =. Then we claim that 


A= z—1+cosz 


1—sinz 


Indeed, let (v, 7) be an ordered pair in which T is a non-plane 1-2 tree 
on vertex set [n] and v is a leaf of T. Then A(z) is the exponential 
generating function counting such pairs. Let us first assume that n > 1, 
and let us remove the root of T. On the one hand, this leaves a structure 
that is counted by A’(z). On the other hand, this leaves an ordered pair 
consisting of a non-plane 1-2 tree with a leaf marked, and a non-plane 
1-2 tree. By the Product formula of exponential generating functions, 
such ordered pairs are counted by the generating function A(z)E(z), 
where F(z) = tan z + sec z is the exponential generating function of the 
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Euler numbers, equivalently, of decreasing nonplane 1-2 trees. Finally, 
if n = 1, then no such ordered pair is formed, while A’(z) has constant 
term 1. This leads to the linear differential equation 


A(z) = E(z)A(z) +1, 
with the initial condition Ag(0) = 0. Solving this equation we get the 
formula we claimed. 


Now we need the following lemma from Complex Analysis. 


LEMMA 10.1 


Let H(z) = ie) be a function so that f and g are analytic functions at 
zo, and f(zo) #0, while g(zo) = g'(z0) = 0, and g"(z) #0. Then 
2 f (zo) 1 h_y 


H(z) = eee Py sete: 
z— 2 


PROOF = The conditions directly imply that g has a double root, and 

hence H has a pole of order two, at zo. In order to find the coefficient 
that belongs to that pole, let g(z) = q(z)(z — 20)?. Now differentiate 
both sides twice with respect to z, to get 


9" (2) = 9" (2)(z — 20)? + 4q/(z)(z — 20) + 24 (2). 


Setting z = zo, we get 


g'' (20) = 2q(z0). (10.6) 
By our definitions, in a neighborhood of zo, the function H(z) behaves 
like 
f() 


and our claim follows by (10.6). | 


Applying this to the dominant term of A(z) with D = 17-2 anda = 7/2, 
we get that 
Wes 2 n+2 


TT 


So now we have the number of all leaves. On the other hand, the total 
number of all vertices is nE,, where E,,, the Euler number. We know 
that E(z) = tanz + secz has a simple pole at zo = 7/2. Using that 
observation, it is straightforward to compute that 


ee (). (10.8) 


n! 7 
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Comparing (10.7) and (10.8), we get that 
2 
lim a, = 1— — & 0.3633802278. 
noo T 


Let A(z) be the ordinary generating function for the number a, of 
all inversions in all such permutations, and let C(z) be the generat- 
ing function of Catalan numbers. Then, it follows from the structure of 
312-avoiding permutations that the differential equation 


A(z) = 2zA(z)C(z) + 22C(z)C"(z) 


holds, with initial condition A(0) = 0. Solving this equation, and then 
computing the coefficients of the obtained power series A(z), we get that 


2 1 2n-1 
oe ne ( ee )+( : ) 
n n—-1 


B(x) = $2 ~ ME nil 


Therefore, 


10.7 Solutions for Chapter 7 


1. 


We know that permutations of length n that avoid 12---k correspond 
to pairs of SYT of the same shape that have at most k — 1 columns. 
Because of Theorem 7.11, involutions of length n correspond to such 
pairs in which the two elements are identical. In other words, 


I,(123---k) => fr, 
F 


where F' runs through SYT on n boxes having at most / — 1 columns. 
The proof is then immediate. 


. These SYT bijectively correspond to northeastern lattice paths from 


(0,0) to (n,n) that never go above the main diagonal. The bijection is 
given by reading the entries in an SYT in increasing order, and if entry 
7 is in row 1, then taking step 7 of the corresponding path to the east, 
otherwise taking it to the north. 


. We claim that P() must be of rectangular shape (10 x 8). Indeed, P(z) 


has ten rows, and its rows are of length eight or less; otherwise, the first 
row would be of length nine, implying that a contains an increasing 
subsequence of length nine. Therefore, the fifth row of P(z) is also of 
length eight. 
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7. 


11. 


13. 


15. 


17. 


19. 


21. 


By the argument seen in the solution of Exercise 1, it suffices to find a 
formula for the number of all SYT on n boxes that have at most two 
columns. In such SYT, the entry n must be at the end of either column. 
If n is even, and our SYT is rectangular, then n must be at the end of 
the second column. This implies the recursive formulae 


2In—1(123) if n is even, 
I,(123) = 
2I,-1(123) = C(n—1)/2 if n is odd, 


where in the last step we use the result of Exercise 3. The result is now 
straightforward by induction as for n = 2m +2, we get 2-(?™*") = 


eee and for n = 2m + 1, we get 2(7"") - ((7"") — ( an y= eae 


m+1 m m—-1 m 


. Theorem 7.5, with its notation, shows that necessarily a, = k and a, + 


a2 +---+a, =7r-k. Thus necessarily a, ag 1) = Op k and 
Gr41 < k, otherwise there would be r +1 increasing subsequences of 
length & that are disjoint. This means that the size of the last column 
is m, = r. Applying (7.1) with k variables instead of k — 1 and fixing 
Mp =T we get the proof exactly as we got the proof of Theorem 7.4. 


In any SYT of shape F’,, the entry n has to be in one of the inner corners. 
So removing n from any such tableaux, we get an SYT of shape F” for 
some F” that is part of the summation. 


Note that with any reductions, the value g(a, b) never increases. So no 
series of reductions can turn q into p. 


Yes, there is. Let ay = 1812 101481169473215. Then let a;,; be 
obtained from 7 by simply inserting two consecutive elements right after 
the maximum element m of a;, and giving them the values (m — 4) and 
(m—1), and of course, relabeling the other elements naturally. It is not 
difficult to see that the a; consist of two decreasing subsequences, and 
that they form an antichain. This construction is due to Miklés Béna 
and Daniel Spielman [52]. 


An isomorphism between the two posets can be constructed using the 
idea given in the proof of Exercise 3 of Chapter 3. 


The statement can be proved by induction on the size of P, the initial 
case being obviously true. See [279], pages 95-97 for the details of the 
inductive step. 


This result is based on an observation from [145]. It is clear that the 
right-hand side is the number of 321-avoiding n-permutations. We claim 
that the left-hand side is the same, counted by the length k& of the 
longest increasing subsequence. Indeed, if 7 is such a permutation, then 
risk(m) has at most two rows, and exactly k columns. There is exactly 
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one Ferrers shape F' satisfying these criteria, and then the hooklength 


formula shows that 
Fem n+l 
n+1 n—k/)’ 


and the proof follows. 


Consider the fraction on the right-hand side of (7.1). Its numerator 
is clearly n!, as it should be. In the denominator, the term (m,_1)! 
is the product of the hooklengths of all boxes in the last row of F. 
How about the term (m,z—2 + 1)! in the denominator? It is almost the 
product of the hooklengths of all boxes in the next-to-last row of F. 
We must say “almost” because if mp_1 < mx_-2, then there will be one 
term missing from the mz_2 + 1 terms (hooklengths) whose product is 
(mp2 +1)!. This is because the rightmost box of row k — 2 that has 
a southern neighbor will have a hooklength that is larger by 2 than the 
hooklength of its right neighbor. Therefore, the missing hooklength will 
be mp_2—™Mz-1 +1. However, this is corrected by the appropriate term 
of the product in the brackets, that is, the term given by 7 = k — 2 and 
j=k-1. 

Continuing this way, we see that the product of the hooklengths in row 
i is 


i<j<k-1 


and the proof follows by taking products for i € [k — 1]. 


Yes, P’ is a lattice. We prove this by induction on n, the initial case 
of n = 1 being trivial. Let us assume that the statement is true for 
n—1, and prove it for n. Let « = x1 %2---@, and y = y1y2°-- Yn be two 
elements of P’. Let x’ (resp. y’) be the (n — 1)-permutation obtained 
from « (resp. y’) by removing the maximal entry n. Let v = a2’ Ay’, 
and let z=2' Vy’. Let x; =n, and y; =n, and assume without loss of 
generality that 7 < 7. It is then easy to verify that inserting n into the 
jth position of v results in x A y, and inserting n into the ith position 
of z results in x V y. 


No, if n > 2, then P’ is not complemented. For instance, let « = 
2134---n. Then y = n(n — 1)---4132 and y’ = n(n — 1)--- 4312 both 
satisfy the requirements. 


No, I, is not self-dual in general. For instance, ifn = 4, then there 
are three elements in J, that cover the minimum element, namely 1243, 
1324, and 2134. At the same time, there are only two elements in I, 
that are covered by the maximum, namely 4231 and 3412. 
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33. 


35. 


37. 


39. 


It is proved in [332] that the number of these posets is 
(1 + o(1))n!? 
2fe 


We have seen that 7 is a descent in Q(p) if and only if 7 is a descent 
of p. However, now p is an involution, so P(p) = Q(p). Therefore, the 
question is reduced to asking what the probability is that 7 is a descent 
of a randomly selected SYT on n boxes. As we have seen in Exercise 24 
of Chapter 6, this probability is 1/2. 


Let fr be the number of Standard Young Tableaux of shape F’. Then 
by the Robinson—Schensted correspondence, and Theorem 7.11, we have 
2 


m= >> fi< fe ihe 


|Fl=n |Fl=n 


On the other hand, the Cauchy—Schwarz inequality implies that for any 
positive real numbers 21, %2,-°-- ,2n, we have 


1 7 
= 


Applying this inequality for the p(n) positive real numbers fr where 
|F'| =n, we get the second part of the claim. 


Consider the elements of S as strings of zeros and ones, given by the 
parity of each entry. Then the elements of S are 0-1 strings of length n 
that contain exactly |n/2] ones. Therefore, if S has more than (ira ) 
elements, then S has two elements p and q whose associated 0-1 strings 
are identical. Then p and q cannot be colliding. 


This elegant argument is due to Janos Korner and Claudia Malvenuto 
[231]. In that paper, the authors mention that for n > 7, a set of (iny2}) 
pairwise colliding n-permutations actually exist. It is not known if that 
remains true if n > 7. 


Both sides count layered permutations. Indeed, the left-hand side counts 
layered permutations, and a layered permutation is automatically an 
involution. 


Da 


10.8 Solutions for Chapter 8 


1. 


This is a classic algorithm for generating permutations, which was found 
independently by Johnson [221] and Trotter [312]. The proof is not 
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difficult by induction on n, the initial case of n = 2 being obvious. Now 
let us assume that the statement is true for n — 1. Then one proves 
from the definition that the algorithm will list the n! permutations of 
length n so that the first n, the next n, the following n, and so on, will 
only differ in the position of n, while the subsequence of the entries from 
[rn — 1] will be unchanged within each of these n-tuples of permutations. 
Among the n-tuples, these subwords will be changed according to the 
list of (rn — 1)! permutations of length n — 1, generated by this same 
algorithm. 


. Such an algorithm can be found in [154]. 


. We have seen that for any n-permutation p, the image s(p) ends in the 


entry n. Iterating this argument (to the shorter string preceding n in 
s(p)), the image s(s(p)) ends in the string (n — 1)n, the image s°(p) 
ends in (n — 2)(n — 1)n, and so on, the image s"~1(p) ends in 23---n, 
so must be the identity permutation. 


. We claim that for n > 3, these are the (n — 2)! permutations of the form 


Snl1, where S is any permutation of the set {2,3,---,2—1}. We prove 
this statement by induction on n. The initial case of n = 3 is obvious. 
Let us assume that we know the statement for n, and let p be an (n+1)- 
permutation that is not (n—1)-stack-sortable. Let p = L(n+1)R. Then 
8(p) = s(L)s(R)(n + 1) is not (n — 2)-stack-sortable. By our induction 
hypothesis, this means that s(L)s(R) = 23---n1. As s(R) has to end in 
its largest entry, we must have s(R) = R = 1, showing that p is indeed 
of the form L(n + 1)1 as claimed. 


. We have seen in Proposition 8.17 that a t-stack-sortable permutation 


must always avoid the pattern 23 ---(¢+2)1. On the other hand, observe 
that Theorem 4.23 implies that 


Avn(234--+(¢-+2)1) = Avy (123--- (t+ 2)) < ¢+1), 
where the last inequality was proved in Theorem 4.21. 


(a) The image of 231 is 123 as shown in Figure 10.7. 


(b) These are precisely the t-stack-sortable permutations. Even more 
strongly, s:(p) = s‘(p) for all p. Indeed, it is easy to see by induc- 
tion on i that the entries of p will leave stack 7 in the order identical 
to s'(p). 


An n-permutation is (n — 2)-stack-sortable except when it ends in the 
string nl. So W,_-2(n,k) is just the number of (n — 2)-permutations 
with k — 1 descents, that is, W,-2(n,k) = A(n — 2,k). As Eulerian 
polynomials are log-concave, our statement is proved. 
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Output Stack 2 | Stack 1 | Input 


231 
y: 31 
2 31 
2 | 3 
‘ 3 
1 5 3 
i2 3 


123 


FIGURE 10.7 
Passing 231 through two stacks. 


15. 


17. 


19. 


Let p = InR, when L and R are allowed to be empty. If neither LD nor 
R are empty, then let f(p) = f(L)nf(R). If L is empty, that is, p = nR, 
then let f(p) = f(R)n. Finally, if R is empty, that is, p = Dn, then 
let f(p) = nf (ZL). Use this same rule recursively to compute f(L) and 
f(R). 


Yes. Take the antichain A from the solution of Exercise 15 of Chapter 7, 
then take the reverse of all the permutations in it, to get the antichain A’. 
Then A’ consists of 321-avoiding permutations, that is, permutations 
containing an increasing subsequence of length at least n/2. Now affix 
the entry 1 to the end of each entry of A’ to get the new antichain A”. 


The condition that pr = qr means that going through the stack has 
the same effect on p and on q. In other words, the movement sequences 
associated to the two permutations are the same. By this we mean the 
following. To each permutation of length n, we can associate a sequence 
of parentheses of length 2n, consisting of n copies of ”(” (left parenthesis) 
and n copies of ”)”, (right parenthesis) describing how the permutation 
passes through the stack. Each time an entry goes DOWN in the stack, 
we write a left parenthesis, and each time an entry comes UP, we write 
a right parenthesis. 


For example, sequence of parentheses of x = 123 is ()()(), the sequence 
of parentheses of y = 321 is ((())), while the sequence of parentheses of 
both p = 132 and g = 231 is ()(()). 
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So p and q satisfy the conditions of this exercise if and only if they 
have the same sequence of parentheses. We claim that this, in turn, is 
equivalent to the condition that T(p) and T(q) are the same as unlabeled 
trees. This is straightforward by induction on n, if we note that the size 
of the right subtree of T(p) and T(q) is k, where there are 2k parentheses 
(k left, & right) inside the pair of parentheses that ends last. Indeed, 
this is just the number of entries that were on the right of n; therefore, 
they entered the stack after n but exited the stack before n. 


We have seen in Exercise 19 that b, just describes what effect the stack 
has on p. We have also seen that this effect only depends on the sequence 
of parentheses associated to p, or, in other words, the unlabeled tree 
obtained from T(p) by omitting its labels. As there are C;, such trees, 
we get that B has C,, = (*")/(n + 1) elements. 


n 


We have seen in Exercise 21 that going through a stack can effect an n- 
permutation in at most C,, ways, so an n-permutation can have at most 
C,, preimages. On the other hand, the identity permutation does have 
C, preimages, namely all the 231-avoiding permutations. We claim this 
is the only permutation with that property. 


To see that no other n-permutation can have C;,, preimages, we apply 
induction on n. For n < 3, the statement is clearly true. Now let us 
assume the statement is true for all positive integers less than n. If 
s(q) = p, and q = LnR, then we have p = s(L)s(R)n. Now keep n fixed 
in q, in position k, and change L and R so that s(q) does not change. 
Note that this means that the set of entries in LZ and the set of entries 
of R cannot change, as otherwise the set of the first k — 1 entries in s(q) 
would change. Similarly, s(Z), and s(R) have to remain unchanged. 


By our induction hypothesis, there are at most Cy_1 ways we can per- 
mute the entries of L without changing s(L), and there are at most 
Cy—k Ways we can do this with R, with equality holding only if both 
s(L) and s(R) are monotone increasing, that is, only if p is increasing. 
As k can range from 1 to n, this means that the number of preimages 
of p is at most 


SS Cy-1Cn—-k = Ch, 
fai 


with equality holding only if p = 123---n. 


(a) These are the permutations that avoid both 2431 and 4231. 


(b) It follows from part (a) and the characterization of ir-sortable per- 
mutations that p is ir-sortable if and only if ((p”)~')" is or-sortable, 
proving our claim. 
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29. 


31. 


An n-permutation p is separable if and only if it avoids both 2413 and 
3142. First we show that this is necessary. Indeed, the statement is 
obvious for n < 4, and follows by induction on n for larger n. 


Now let p avoid both 2413 and 3142. Then clearly, the reverse of p also 
avoids these patterns, so we can assume without loss of generality that 1 
precedes n in p. Then all entries on the right of n have to be larger than 
all entries on the left of 1, or a 3142 pattern would be formed. If there 
is no entry on the left of n that is larger than an entry on the right of n, 
then p is separable, and we are done. Otherwise, let d be the leftmost 
entry on the left of n that is larger than the smallest entry c on the right 
of n. Then dnc is a 231-pattern, meaning that there cannot be any entry 
b < c located between d and n, for dbnc would be a 3142 pattern. So 
all entries located between d and n are larger than c, whereas all entries 
on the left of d are smaller than c, proving that p is separable. Indeed, 
p = LR, where the split occurs immediately before d. Then both L and 
R avoid 2413 and 3142, and therefore, are separable. 


The concept of separable permutations was introduced in [85]. 


These are the permutations avoiding the pattern q = (¢ + l)t---1. 
It is clear by the Pigeonhole Principle that these permutations cannot 
be t-queue-sortable. On the other hand, if p avoids q, then let us 
define the co-rank of an entry to be the length of the longest decreasing 
subsequence ending at that entry. Then p is t-queue-sortable by sending 
all entries of co-rank 7 to queue 2. 


We claim that this number is 


Ce) 


_ on ic fet 
PU 2 ws )ee: 


The result was first announced in this form by Petter Brandén [94], with 
a computational proof. A combinatorial proof can be given as follows. 
Let us identify stack-sortable permutations with their decreasing binary 
tree, which, in this case, can be viewed as an unlabeled plane tree. Let 
us take an unlabeled binary plane T tree on & vertices. If a vertex v 
of T has less than two children, add new children as needed until all 
the vertices of T have two children. Let T’ be the tree obtained in this 
way. Then T”’ has k peaks and k+ 1 leaves; hence T” has 2k edges. Now 
subdivide some of the edges of T’ by new (blue) vertices until the total 
number of edges of the obtained tree T” is n — 1. (The new vertices can 
also be added on the left of the leftmost vertex of T” and on the right 
of the rightmost vertex of T”.) Now T” has n — 1 — 2k vertices that 
have only one child; there are 2”~!~?* ways to rearrange the left-right 
orientation of these edges without changing the structure of the non- 
blue vertices. As there are (*)/(k + 1) choices for T, there are ("),') 


k 2k 
ways to turn J” into T’’, and the statement is proved. 
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Setting ap = 0, we have A(z) = 0,59 @n2” = z/(1 — 2z— 7). The key 
observation is that in permutations counted by a,, the entry n has to 
be in the first, second, or last position. For details, and many similar 
results, see [156]. 


Let + have fertility A, and let 7 have fertility B. Then we claim that 
the permutation 
n=(yOor)@l 


has fertility AB. 


For instance, 213 has fertility one, while 123 has fertility five, so our 
claim in this case is that 5461237 has fertility five. 


It is clear that 7 has fertility at least AB, since if s(p) = y and s(¢) = 7, 
and ¢ = (9p @1)6¢, then s(¢) = z. In the above example, 7 = 5467123. 


Now we show that no other permutations are preimages of 7. Note that 
in 7, all entries coming from y precede all entries coming from 7. As 
the former are larger, it follows that in any preimage of 7, there must 
be an entry between the leftmost of the entries that belong to y and the 
rightmost of the entries that belong to 7. By exclusion, that entry must 
be the maximum entry, and our claim is proved. 


DT 


10.9 Solutions for Chapter 9 


1, 


First, let n be odd. Taking an alternating path that starts with the gray 
edge from 0 to 1, then continues with the black edge from 1 to 2, and 
so on, we reach the vertex n with a gray edge, after which we must go 
to 0 using a black edge, completing an alternating cycle. The remaining 
edges form the other alternating cycle. 


Now let n be even. Take the alternating path that starts as the one in 
the previous paragraph. This path will reach the vertex n with a black 
edge, after which we must go to the vertex n+ 1 using a gray edge, 
then to 1 using a black edge. After this, we walk through the vertices 
1,2,--+,n again, using the edges that have not been used yet, finishing 
with the black edge from 1 to 0. This shows that all of G(p) is one 
alternating cycle. 


We claim that the canonical block interchange defined in the proof of 
Theorem 9.9 has this property, with « = p;. Recall that x is defined 
as the smallest entry of p for which there exists an entry y in p that 
precedes x and is larger than x. 
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11. 


13. 


Indeed, if p does not start with the entry 1, then x = 1, and z has the 
property required in this exercise. If p = pip2---Pn, and p; = j for 
j <k, but py #4 k, then the smallest entry p; for which p; 47 is k. This 
entry & is not in the kth position, so the entry in the kth position is 
larger than k. So k is eligible for the role of x, and no entry less than k 
is eligible for the role of x, since the entries that are smaller than k are 
in increasing order at the beginning of p. 


A block transposition removes three black edges in G(p), and replaces 
them by three new black edges. The old black edges could be part of 
one, two, or three cycles, of which none, one, two, or three could have 
been odd, and the new black edges are also part of one, two, or three 
cycles, of which none, one, two, or three could be odd. The only way in 
which this would result in coga(G(p)) growing by more than two would 
be if the three new black edges would be in three separate odd cycles, 
while the three old black edges would have been in three separate even 
cycles. However, that is impossible for parity considerations. 


Let us assume without loss of generality that the alternating path P 
from u to v starts with the gray edge from u to u+ 1 and ends in the 
gray edge from v — 1 to v. We show that P can be completed to an 
alternating cycle. At each vertex vu; starting at vi = v, use the edge 
leaving v; whose color is different from the edge that we used to arrive 
at v;. This creates an alternating path that eventually gets back at u 
with a black edge since the graph is finite and no vertex is the endpoint 
of more than one edge of each color. 


There is a natural bijection between the cycles of the two graphs. If C 
is a cycle of B(p), consisting of the vertices 


Da, Ray—1 La, Ray~1 ao Loa, Raz—1) 


then Ra,-1La,,, is a black edge in B(p), implying that a; — 1 is the 
entry immediately preceding a,;+41 in p. However, that means that there 
is a black edge from a;+1 to a; — 1 in G(p), and so 


Qk (@k—1 — 1) ag—1 (Qk—2 — 1) ag—2-++@1 (ax — 1) 
is a cycle of G(p). 


Create the permutation p’ = 897564231 with the first move. Then note 
that due to three entries followed by entries one larger, sorting p’ is 
equivalent to sorting the decreasing permutation of length six, and we 
have seen in the text how to achieve that with four block transpositions. 


Create p’ = 963785241, then p’” = 967852341 with the first two moves. 
Then proceed as in the solution of Exercise 11. 
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15. If p is not the increasing permutation, then the value of is(p) can be 
increased by one by a block transposition. Indeed, find the longest 
increasing subsequence s of p, find an entry x ¢ s, and insert x into the 
right position of s. An example for permutations for which this bond is 
tight is 

p=246--- 2n135---(2n—-1). 
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n(n—1)---(n—m-+1) 

number of partitions of the integer n 

set of all n-permutations 

number of n-permutations avoiding the pattern q 

number of n-permutations containing exactly r copies of q 
number of partitions of the set [n] into k blocks 


(—1)""*e(n, k) 


image of the permutation p under the stack sorting operation 


number of alternating cycles in the alternating cycle 


decomposition of the cycle graph G(p) of the permutation p 


e c(T'(p)) 


number of cycles of the permutation p 


e btd(p) number of block transpositions needed to turn the permutation 
p into the identity permutation 
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pop-stack-sortable, 368 
quasi-Stirling, 46 
reverse alternating, 34 
reverse of, 151 
separable, 383 
simsun, 42 
skew decomposable, 174 
skew indecomposable, 174, 222 
skew-merged, 202 
smooth, 448 
sorted, 382 
stack-sortable, 352 


506 Combinatorics of Permutations, Third Edition 


t-, 355 

Stirling, 41 

strongly g-avoiding, 201 

two-series-sortable, 385 

uniquely sorted, 386 
Permutation class, 150, 354 

principal, 150 
Permutation matrix, 87 
Pevzner 

Pavel, 401 
Polynomially recursive sequence, 39 
Pop-stack Sorting, 368 
Pop-stacks in parallel, 386 
Poset 

graded, 320 

naturally labeled, 50 

self-dual, 328 

two-dimensional, 341 
Postorder, 362 
Prefix exchange, 419 
Probability, 257 

conditional, 267 
Probability generating function, 293 
Product formula, 114 


g-log-concave polynomial, 81 
g-multinomial coefficient, 70 
g-unimodal polynomial, 81 
Queue, 375 

parallel, 383 


Random variable, 273 
independent, 273 
indicator, 276 

Rank 
of a vertex, 279 
of an element of a poset, 321 
of an entry, 157 

Rank-generating function 
of a poset, 341 

Real zeros property, 21 

Reduced decomposition, 324 

Reduction, 320 

r-fall, 37 

Riffle shuffle, 298 


Right-to-left minimum, 329 
Rise, 382 
risk, 311 
Robinson-Schensted—Knuth correspon- 
dence, 309 
Row insertion, 316 
Run 
alternating, 24 
ascending, 3 
very tight, 44 


Sample space, 257 
Schlomilch’s formula, 130 
Schroder numbers, 208, 446 
Schiitzenberger 

Marcel-Paul, 317 
Semifactorial, 119 
Shape—Wilf—equivalence, 209 
Shareshian 

John, 82 
Simion 

Rodica, 151 
Simplicial complex, 42, 331 
Skew sum 

of permutations, 157 
Spine structure 

of a 132-avoiding permutation, 

302 

Stack 

in series, 383 
Stack sorting, 351 
Stack sorting algorithm 

left-greedy, 382, 384 

right-greedy, 357, 386 
Stack word, 357 

of a 3-stack-sortable permutation, 

357 

Standard deviation, 290 
Standard Young Tableau, 259 

partial, 316 
Stanley 

Richard, 6, 30, 411 
Stanley—Neggers conjecture, 45 
Stanley—Wilf conjecture, 155 
Stanley- Wilf limit, 156 


Index 


Statistic 
Euler-Mahonian, 80 
Stirling number 
l-, 43 
of the first kind, 103 
—, 134 
signless, 97 
of the second kind, 10 
Subsequence of a permutation, 
Sum 
of permutations, 156 
Superpattern 
weak, 250 
superpattern, 247 
Superpermutation, 249 
Symmetric group, 85 


Tableau 

balanced, 324 
Total relative displacement, 48 
Transition lemma, 110 
Transposition 

adjacent, 131, 323 
Transpositions, 93 
Tree 

(0,1), 173 

B(1, 0), 385 

binary plane, 197 


decreasing binary, 39, 278, 


361 


507 
Wilf-equivalence, 154 


Zeilberger 
Doron, 82, 355 
Zigzag word, 254 


149 


302, 


decreasing non-plane 1-2, 37, 300 


increasing, 137 
minmax, 39 


ULIS, 201 
Unimodal sequence, 15 


Valley, 24 
Variance, 288 


Wachs 

Michelle, 82 
Well-quasi-ordering, 330 
Wilf 

Herb, 259, 266, 306 


